Built in 1812, it was the home of Dr. Francis Julius LeMoyne (1798–1897), an antislavery activist who used it as a stop on the Underground Railroad.

LeMoyne also assisted in the education of freed slaves after the American Civil War, founding the historically black LeMoyne–Owen College in Memphis, Tennessee.

In 1834, LeMoyne became active in the abolitionist movement, speaking and organizing on behalf of political solutions to the problem of slavery.

Although he was not a radical firebrand in the mold of William Lloyd Garrison, LeMoyne was a regional director of antislavery societies until about 1850.

It undertook to restore the building to a state similar to that during LeMoyne's lifetime,[4] and now operates it as a historic house museum, offering guided tours year round.