[1] The main structure, a substantial three-storey rectangular pele tower with rounded turrets and battlements, was constructed about 1370, and was the home of the Middleton family.

In 1614 Thomas Middleton built a new manor house attached to the tower.

[1] A west wing was added in 1711 but was largely demolished in 1872 by Sir Arthur Middleton when the remainder of the house was considerably altered.

[2] The castle was abandoned as a residence by the family in the early 19th century when Sir Charles Monck built Belsay Hall close by.

[3] The interiors were largely removed and it was then used as a ready-made folly, as was fashionable among the aristocracy at the time, serving as the setting for garden parties and other entertainments.