It was a stop on the Underground Railroad, a network of people and places that facilitated the movements of escaped slaves.

The AIA Journal called it the "Oldest standing concrete structure of any consequence.

Goodrich was a Seventh Day Baptist from the Burned-over district of western New York, and like most people of his faith, he was known for his anti-slavery sentiments.

The house's proximity to these routes and the Rock River allowed it to function as a stop on the Underground Railroad; fugitive slaves in Wisconsin frequently followed the river to the area before taking the road to Racine, where a boat could take them to Canada.

Goodrich also hosted prominent abolitionist Sojourner Truth at the house when she visited Milton in the 1860s.