After the Norman Conquest Chaffcombe was granted to the Bishop of Coutances under whom it was held by Ralph le Sor until it was acquired by Oliver Avenel (died 1226) and inherited by his descendants until 1613 when John Poulett of Hinton St George became lord of the manor and stayed in the Poulett family until 1913.

Avishays on Whiteway Hill is a large detached house which was partly built in the 17th century, but substantially altered between 1745 and 1759.

[9] It stands on the site of a former sham castle which had fallen into ruin, and served as a water tower as well as having a clock on the west side facing the house.

The clock, which has a hand wound mechanism, was built by Gillett and Johnson of Croydon and was originally at Highclere Castle in Hampshire.

[11] The Anglican parish Church of St Michael and All Angels has a tower which partly dates from the 15th century, the remainder being rebuilt under J. M. Allen between 1857 and 1860.