It is also controversial whether the Romans maintained a military presence to protect the ford, nearby, across which the convoys of lead from Lutudarum (possibly Wirksworth) joined Rykneld Street at Derventio (now part of Derby), en route for the North Sea ports.
However, remains that appear to be of Anglo-Saxon origin have been found, suggesting occupation by persons of some position, possibly a Saxon Thane of the name of Siward, or his relatives.
In or around 1066 Henry de Ferrers (sometimes spelt Ferrars), having rendered great service to King William, was granted estates in Derbyshire, which became known as Duffield Frith.
His great grandson, William, who succeeded to the position in 1162 joined the King's sons in a rebellion against their father, Henry II, and in 1173 both castles were destroyed.
Furthermore, H. Walton also believed that the second floor contained domestic and small apartments, kitchens, private retirement rooms and possibly a chapel.
Although further investigations were carried out in the 1930s and in 1957, few medieval remains were found, but the idea that the site had been occupied before the arrival of the Normans was confirmed by a number of Romano‑British finds.
A geophysical investigation of the site was carried out by the University of Bradford in 2001 and this revealed traces of other structures to the south and the southeast which would appear to date from the same period.