The castle may have been built by Eustace II, Count of Boulogne, who obtained the manor of Ongar in 1086.

A stone tower was built in 1150[1] and was visited by King Henry II in 1157, when it was held by Richard de Lucy.

[2] A stone keep was built on top of the motte around 1086, which was confiscated by Richard III in November 1483[3] after he had Henry Stafford, 2nd Duke of Buckingham executed for treason, and then was later given by Henry VIII to William Morice in 1541, but this was pulled down by Morice sometime between 1558 and 1579, and he had it replaced by a brick building,[3] itself destroyed in the 18th century.

The motte itself is now covered with trees and is in private ownership, but can be seen from a public footpath that starts at the north end of the High Street.

[citation needed] In 1881, excavations on the site discovered four flint arches and a large block of masonry which probably supported by a post a flooring above; a Saxon spearhead was also discovered.