[1] The manor house was built in 1284 by Robert Burnell, Bishop of Bath and Wells, friend and advisor to King Edward I.

Robert Burnell was granted a royal licence to crenellate and fortify the manor on 28 January 1284, a benefit only extended to trusted people.

[2] Before this, in the autumn of 1283, Edward I had held a Parliament at Acton Burnell, presumably in the adjacent great barn,[2] the only building large enough.

[3] When Robert Burnell died in 1292, the estate was passed down through the family line, eventually becoming owned by the Lovels of Titchmarsh, via a marriage.

Following the Battle of Stoke Field in 1487, the land was confiscated by Henry VII, who in turn granted it to Thomas Howard, 2nd Duke of Norfolk.