The name of the community was changed to Chesterfield in 1888 by congressman Anthony Bullock.

The oldest part of the Recklesstown Tavern, now the Chesterfield Inn, was built c. 1710.

It was added to the National Register of Historic Places on August 19, 1975, for its significance in agriculture, religion, and transportation.

The Kessler House is a two and one-half story brick building built c. 1849.

The Chesterfield Baptist Church was built in 1848 and features Greek Doric columns.