The site upon which Yattendon castle stood was originally occupied by a moated manor house.

[1] His daughter, Alice, married Sir John Norreys of Ockwells (d. 1 September 1466), a Knight of the Shire for Berkshire, and keeper of the wardrobe for Henry VI.

[1] Sir John bought many neighbouring estates and received a Royal licence to crenellate the manor house on 20 January 1448 and to empark some 600 acres (2.4 km2).

[1] John and Alice's son, Sir William Norreys (1433 – 4 January 1507) later inherited the castle.

[2] The castle was probably the residence of William's eldest son Sir Edward Norreys (d. 1487) during his father's lifetime.