During the American Civil War, both Lacy brothers became Confederate officers, and caretakers managed the property when it was not being used by the armies fighting nearby.

Before the Battle of the Wilderness the following year (which proved a costly draw for Confederate forces), Union general Gouverneur K. Warren used Ellwood as his headquarters.

By modern times, the house became the only structure surviving from the latter battle, and was ultimately donated to the government in 1977 and became part of the same Fredericksburg and Spotsylvania National Military Park as Chatham Manor.

During the American Revolutionary War, Churchill Jones rose to the rank of Major in the Southern Legion, under the command of "Light-Horse Harry" Lee.

In 1781 Ellwood hosted troops of the Marquis de Lafayette as he waited for reinforcements from General Anthony Wayne on their way to what became the final campaign at Yorktown.

[2] In the 1860 census Horace Lacy had owned upwards of 249 slaves as well as about 49,000 acres of land (including roughly 6000 at Chatham), with the vast majority used at Ellwood, depending on the season.

[3] Most worked as field hands or house servants, but Lacy also employed skilled tradesmen such as millers, carpenters, and blacksmiths.

Betty Lacy helped found the Ladies Memorial Association of Fredericksburg, establishing the Confederate Cemetery, and her husband traveled and made speeches to raise money.

After Betty Lacy died in 1907, Indiana University law professor Hugh Evander Willis acquired the Ellwood property.