Late in the 12th century a rectangular tower was built on an earlier enclosure of earthworks by Richard Fitz Gilbert, founder of the de Clare family.

[2] The Surrey branch of the medieval noble Clare family lived in the village and sided with the barons during the disputes with the Crown in the 13th century, and in the wars of 1263–4 it was, so far as is recorded, for the first and only time was the scene of military operations.

H. E. Malden states with citations: "it is probable, however, that the castle was not totally destroyed by this comparatively small force, but that, having once been dismantled, it fell into neglect and became gradually ruined.

[3] The site has been quite overgrown for many years but vegetation does not cover part of the minority of materials that remain, as walls, doorways and arches.

Specifically, one outer ditch has been partially infilled and the site has been indented by foundation-laying, associated water pipes, drain, electricity cable and access road for the 19th century construction of the Castle Hill home and now separately owned Stable House and Garden Cottage near the centre of the large mound, its ringwork and bailey (fortification) at Bletchingley survives well and "large areas, especially within the ringwork, lie apparently undisturbed".

The ditch is spanned on the NE side by a causeway 3m wide which marks the original access route into the inner part of the castle.

Between the inner and outer defences was the bailey, where ancillary buildings such as stables and storage huts were sited.Towards the north-west is a small mound which may mark the site of a barbican.