It was built by Robert de Umfraville in 1076, not long after the Norman Conquest and stands on a natural spur of a hill.

[2] It was abandoned by the mid-13th century, and is not mentioned in the estate of Gilbert de Umfraville when he died in 1245.

This may be a reference to Siward the Dane, earl of Northumberland in the reign of Edward the Confessor.

[5] In the 18th century, excavation around the side revealed a Roman stone, which may have come from the fort at High Rochester and re-used for building.

[4] Elsdon Tower is nearby: a pele tower dating from the late 14th or early 15th century,[6] which may also have been built by the de Umfravilles, though for six hundred years it was the Elsdon Rectory until 1961, when it was converted to private buildings.