His grandfather Thomas Lee (1690-1750) had considerable acreage in what was or became Prince William, Fairfax and Loudoun counties before Ludwell was born.
However, their mother died after giving birth to three more daughters, Mary and Hannah (who would both marry members of the Washington family), and Marybelle (who did not reach adulthood).
Meanwhile, these two elder brothers were sent to London, England, where their merchant uncle William Lee lived with his wife and decade younger children.
[6] Lee and schoolmate and soon-to-be brother-in-law Bushrod Washington scouted as the company harassed British Banastre Tarleton, who was raiding Southern plantations as far away as Albemarle County.
Ludwell Lee's father died in 1794, burdened by debts such that two auctions were made of his property, and his namesake grandson would publish two volumes of his grandfather's memoirs to rescue his name and honor.
[17] A determined Federalist, Lee concluded his legislative service with several terms as state senator representing Fairfax and Prince William Counties.
Upon moving to Loudoun County in 1800, Ludwell Lee ended his political career rather than challenge multi-term delegates William Noland and Joseph Lewis, or longtime senator Francis Peyton.
[25] Lee died in 1836, survived by his second wife, who with the assent of his children, sold Belmont Manor to Margaret Mercer, a dedicated member of the American Colonization Society who had worked as a teacher in order to pay her father's debts, free the slaves she had inherited, and send them to Africa a decade earlier.
She operated a school for girls at Belmont, and tried to use the plantation to demonstrate that farming could be successful without enslaved labor, although after her death her executors sold the property to Alexandria's largest slave trader.