[2] The initial castle comprised two sets of buildings, linked by a small courtyard and protected by a curtain wall on both sides, surrounded by a large, circular, water-filled moat approximately 7.4 metres (24 ft) wide, and an inner moat, since largely destroyed.

[3] The male Tilliol line died out in 1435; the castle then passed by marriage to the Colville family.

[4] They rebuilt much of the castle in the late 15th century, including remodelling the pele tower, the great hall and the gateway, complete with a polygonal barbican.

[8] In February 1645, Parliamentary forces besieging nearby Carlisle also besieged and eventually seized Scaleby Castle, causing considerable damage; Edward recovered the castle, but in at the start of the Second English Civil War in 1648 he again took up arms on behalf of the king.

[10] Sir Edward was heavily in debt so he sold the castle after the war to Richard Gilpin, who restored the property c. 1800.