[7] The Chesterfield Spiritualist Camp District and George Makepeace House are listed in the National Register of Historic Places.

It is bordered to the west by the city of Anderson, the Madison county seat, and to the east by the town of Daleville.

Indiana State Road 32 passes through Chesterfield as its Main Street, leading west into Anderson and northeast through Daleville 13 miles (21 km) to Muncie.

Interstate 69 passes just east of the Chesterfield town limits, with access from Exit 234 (SR 67).

The White River flows east to west along the northern border of the town.

According to the U.S. Census Bureau, Chesterfield has a total area of 1.37 square miles (3.55 km2), all land.

[1] As of the census[10] of 2020, there were 2,490 people, 1,098 households, and 404 families living in the town.

54.3% of Chesterfield residents were employed, working a mean of 38.6 hours per week.

The median gross rent in Chesterfield was $887 and the homeownership rate was 64.6%.

As of the census[11] of 2010, there were 2,547 people, 1,090 households, and 663 families living in the town.

32.7% of all households were made up of individuals, and 11.8% had someone living alone who was 65 years of age or older.

As of the census[5] of 2000, there were 2,969 people, 1,269 households, and 808 families living in the town.

30.8% of all households were made up of individuals, and 11.6% had someone living alone who was 65 years of age or older.