It was built shortly after the Norman Conquest of England in 1066 by Richard Fitz Gilbert, having high motte and bailey and later improved in stone.

In the 14th century it was the seat of Elizabeth de Clare, one of the wealthiest women in England, who maintained a substantial household there.

Soon after the Norman conquest of England, Richard Fitz Gilbert was granted a barony by William the Conqueror, with two blocks of land, first in Kent and later across Suffolk and Essex.

[6] The castle was built on the site of a former Anglo-Saxon manor house, probably reflective of the wider Norman effort to demonstrate that their authority had replaced those of the previous lords.

[18] The castle itself had four stone towers protecting the entrance to the inner bailey and the keep, called Auditorstower, Maidenstower, Constabletower and Oxfordtower.

[18] The castle's three parks continued in active use, and as part of the breeding programme, the deer were moved between them as they grew older.

[23] The staff at Clare Castle included falconers, tailors, chaplains and goldsmiths, supported as necessary by 30 knights and squires.

[16] The subsequent years saw the Mortimers heavily involved in the Wars of the Roses; after Edmund's death in 1425, the castle passed to Richard of York and in turn, via his son Edward IV, to the Crown.

The masonry of the castle had probably been stripped for use as local building materials, as this part of England was traditionally very short of suitable stone.

[16] After Mary, the castle was acquired by Sir Gervase Elwes of Stoke by Clare,whose family retained it until the 19th century.

[3] In 1867 the Cambridge and Colchester branch line of the Great Eastern Railway was built through the castle, cutting across and largely destroying the inner bailey in order to make room for a new station.