Archaeological excavations show that it was one of the first Norman castles in England to have stone-built fortifications, with more sophisticated defenses being added in later centuries.

Further investigation found similarly early curtain walls defining a small triangular bailey.

It is now thought that the castle did not have an initial timber phase, and that it was built in stone from its foundation in 1067 by William FitzOsbern, 1st Earl of Hereford, and completed before his death in 1071.

In 1401, John Chandos was ordered by Henry IV to refortify his castle against the Welsh Revolt of Owain Glyndŵr.

In 1540, the area was visited by the antiquary John Leland who described the castle as follows:[3] There is a castell a mile and more benethe Dorston apon the right ripe of Dour.

By the mid-17th century, the manor and castle had gone into the hands of the Prosser family and their relatives, the Powells, who held the property until it was sold in 1940.

Work commenced on clearing vegetation in 2016, and consolidating the masonry with grants from Historic England in 2017 and the Garfield Weston Foundation in 2019.

[7] The castle is in the northwestern end of the Golden Valley in Herefordshire, and sits on the summit of a small and steep sided hill between the River Dore and its tributary, the Snodhill Stream.

Polygonal in plan, the base of the tower appears to have been thirteen-sided, but above the first floor level, the surviving southern section suggests an irregular ten-sided construction required to make the best use of the motte's summit.

This change in configuration was the result of an early collapse of the keep, and there are indications of multiple build and repair phases.

The west front was later remodeled by the addition of a solid drum-shaped tower in the southwest, and it is presumed that a second one was built in the northwest, although no discernible trace of this was found during excavations.

There is a shallow ditch on the east side of the motte, the main bailey being surrounded by unusual terraces.