It was a 12th-century stone motte and bailey fortress that, according to tradition, was founded by Eustace II, Count of Boulogne.

It passed into the hands of the de Anstey family in the middle of the 12th century and was strengthened during the First Barons' War of 1215–1216 by Nicholas de Anstey, an opponent of King John, fighting for the barons.

[1][2][3] After the death of de Anstey in 1225, the castle and the surrounding estate was seized by the king from his daughter who was a minor at that time.

He tasked William Fitz Baldwin to acquire the estate and deliver it to Robert de Rokele, who shall keep it on behalf of the Bishop of Canterbury.

A large mound also remains on the site, which was excavated in 1902–1903 by R.T. Andrews and W. B Gerish but their findings about the age of the castle and when it was built were inconclusive.