[1] The building was originally commissioned as a shambles, i.e. a meat market, and the site chosen for it was to just to the south of the Butter Cross which itself was rebuilt in 1673.

[2] The lord of the manor at the time was Sir Stephen Fox of Redlynch Park, courtier to King Charles II, and the "richest commoner in the three kingdoms".

[3] The shambles was designed in the neoclassical style, built in rubble masonry with hamstone dressings and formed part of a trio of new buildings, including a tolsey house (i.e. a building for administering market tolls) and a storage shed (used for storing sheep hurdles), which were provided for the use of local merchants under a lease of markets and fairs, in 1688.

[1] The assembly room on the first floor was used for public events such as the annual meeting of the Mid-Somerset Labourer's Friend Society, chaired by the local member of parliament, Sir Alexander Hood, in October 1850.

[1][8][9] Ownership of the building was passed down various generations of the Fox family until Giles Fox-Strangways, 6th Earl of Ilchester sold it to the parish council in 1913.