[2] Surveyor Robert Caldwell settled at the site of the present house in 1809, becoming the first settler in the vicinity.

His son Samuel expanded the farm and began to diversify, building both a gristmill and a sawmill on his property by 1836.

[3] Soon after the county's establishment, Caldwell agreed to donate land on his farm for public purposes; in 1857, part of this land was chosen as the site of the new county seat, and he was made the namesake of the new village.

[2] Built of brick on a stone foundation, Caldwell's house is a two-story structure constructed in an undeterminate architectural style.

Four lesser but related buildings on the same property were also designated as historic as part of the process.