At an early age, his father began to call him Rooney; what prompted him to use this nickname is not known, but it stuck as a way to differentiate him from his cousin Fitzhugh Lee.

In 1859, Lee resigned from the U.S. Army to operate his White House Plantation, on the south shore of the Pamunkey River, in New Kent County, Virginia.

Loring's forces were transferred to the lower Shenandoah Valley and the command of Stonewall Jackson in late 1861 and occupied the town of Romney in early 1862.

During this time, Rooney's nearby White House plantation was burned to the ground, and his son Robert died of typhoid fever.

During the Northern Virginia Campaign, Rooney played a leading role in Stuart's well-crafted attack on General John Pope's supply base at Catlett's Station on August 22, 1862, capturing a paymaster's safe full of Yankee greenbacks.

[3] Upon his recovery, he temporarily commanded Fitzhugh Lee's cavalry brigade in Stuart's Chambersburg Raid, his conduct earning him promotion to brigadier general.

As a prisoner of war, he was sent to Fort Monroe for several months, before being shipped to New York, where he was held until returned to the Confederate Army on February 25, 1864, in exchange for Union Brig.

[4] In April 1864, Lee was promoted to major general and commanded a division in the Cavalry Corps during the battles of The Wilderness, Todd's Tavern, Spotsylvania Court House, and North Anna in the Overland Campaign.

Lee's cavalry division patrolled the extreme right of the Confederate lines during the Siege of Petersburg, defending against the Wilson-Kautz Raid at Staunton River Bridge, Sappony Church and First Ream's Station.

After their mother died in 1873, Rooney inherited Ravensworth Plantation, the old Fitzhugh family property (near present-day Springfield) in Fairfax County with 563 acres (2.28 km2) of land.