It is located south of Owen Sound and north of Durham where Highways 6 and 10 merge.

Originally named Johnstown after an early landowner, the post office was renamed Holland East in 1851.

The name comes from Chatsworth House, in Derbyshire, near the home town of the postmaster at that time.

[2] Chatsworth was founded in 1848 at the northern terminus of the Toronto-Sydenham Colonization Road.

The Toronto, Grey and Bruce Railway once passed through the village, but that section of its line (then owned by Canadian Pacific) was abandoned in the 1980s.