[2] The exact date of the site's construction is unknown as well as its original purpose, however estimates suggest it was built some time between 1175 and 1350 and contained a moat with high embankments.

Sir Hugh Hulse and his wife Marjory are recorded as the hall's first occupants with documents showing the granting of permission for the construction of a tower on 11 February 1398.

In a vaulted chamber at the base of the tower, which was possibly used as a chapel, there is a roughly-cut corbel which resembles a smiling cat, although it may actually be a red lion rampant.

In his memoirs, Lewis Carroll wrote that he "saw a Cheshire cat with a gigantic smile at Brimstage carved into the wall".

This is therefore believed to be the inspiration for the Cheshire Cat character in his book Alice's Adventures in Wonderland.