The first residents of Chilton were African-American former slave Moses Stanton and his Native-American wife, Catherine, who arrived in January 1845.

[6] John Marygold bought the place in 1852 and, according to legend, called it "Chilington," referring to Chillington Hall in England.

[7][8] He sent a verbal message to have the name change recorded in Stockbridge, then the county seat.

Because the middle ing in the name was accidentally omitted, the municipality was recorded as Chilton.

[7] An alternative explanation for the name is that it was a reference to one of two villages called Chilton near Oxford, England.

"Chilt," when used in a place name in England, is thought to have derived from the Old English word for chalk or limestone ("cealc"),[14][15] while "ton" derives from the Old English word for town ("tun").

[17] Oxford being just 80 miles from Birmingham,[18] it's plausible that Marygold had family members there and knew of the two nearby Chiltons.

Chilton annexed the unincorporated community of Gravesville[20] in the late twentieth century.

[21] Graves built a sawmill in 1849 which remained in the community until he moved it to Fond du Lac in 1886.

[21] At its peak, the community had several general stores, a furniture factory, and a saw/planing mill.

34.7% of all households were made up of individuals, and 16.7% had someone living alone who was 65 years of age or older.

A widening of the river called Lake Chilton has been used for ice skating in winter.

[30] There are 5.3 primary care physicians per 100,000 population in Chilton compared to the statewide average of 75.6.