It is thought to have been originally built around 1370 for William Latimer, the sheriff of Somerset and Dorset, after the manor passed to him in 1355.
[4] The present day site—set in a T-shape[2]—comprises a two-storey solar and half of the hall to the east of that, both with open timber roofs;[1] the foundations of the west range and an extension of the hall are now visible only as earthworks.
[5] The house is unusual among its class of building in retaining many of its original features;[5] this despite having undergone many alterations during its 600-year history.
"[1] In the sixteenth century the solar wing was extended to the north and the hall remodeled by architect Thomas White and his wife Anne;[1] the house remained in the White family until at least the time of Charles I.
[5] After the Restoration, it was bought by Sir Thomas Freke and retained by his descendants, the Pitt-Rivers family; they reduced the length of the hall by 2 metres (6.6 ft), it having previously extended almost to the River Stour, and added a new fireplace and ceiling.