[4] From the 13th century, it was used as a toll house for herring catches from the quay[2] and is believed to have first been called The Tolhouse in 1360.

[4] In 1261, King Henry III gave permission for the Tolhouse to be used as a jail (gaol).

[8] In 1645, witch-hunter Matthew Hopkins used the Tolhouse jail during his witch trials, in which several local women were hanged.

[2][8] In the 18th century, the Tolhouse used hard labour, on a treadmill or picking oakum nearby.

[8] In 1864, HM Inspectorate of Prisons criticised living conditions in the jail, as five people were sharing one room.

[2][7] The Tolhouse was bought by the local authority in 1552,[1] following which significant repair work was undertaken.