Her husband had suffered repeated financial reverses, characterized by poor investments, and was forced to declare bankruptcy in 1809 and move to debtor's prison.

In a July 1812 outbreak of political violence in Baltimore, War Hawk rioters raided a Federalist strongpoint and tried to lynch its defenders, including Harry Lee.

[9] Lee emigrated to the South Atlantic coast and the Caribbean in a series of futile attempts to find a place to recuperate from his injuries.

[1] As a widow and the head of her household, Anne Carter Lee brought up her surviving children during her remaining eleven years of life.

Her medical diagnosis, if any, is unknown, and her health challenges were primarily recorded as an additional obstacle to the young life of her fourth son Robert.

Although not poor, she was not able to face college tuition bills for Robert, who therefore sought higher education opportunities at the United States Military Academy at West Point.

She clung to life until the graduation of her son, and was given a place to live and be nursed at the home of an uncle, William Henry Fitzhugh, the Fairfax County plantation of Ravensworth.