Built in 1808, it is the oldest surviving building in Ryegate, and a distinctive example of stonework by Scottish immigrants.

[1] The Whitehill House stands in a rural area of northwestern Ryegate, set back about 450 feet (140 m) on the west side of the Groton-Peacham Road.

[2] The house was built in 1808 by James Whitehill, an immigrant from Inchinnan, Renfrewshire, who arrived in Ryegate in 1798.

Whitehill was one of a large number of Scottish immigrants to the region, and was one of the founders of the local Presbyterian church.

The house remained in the hands of descendants of Whitehill until 1928, when it was transferred to a family association.