His father, Thomas Sim Lee was a prominent planter, patriot and politician, who had twice served as Maryland's governor and held various other offices.
Lee received a private education appropriate to his class, then traveled to Cambridge, Massachusetts, where he studied law at Harvard University.
[4] Lee did not practice law, but rather primarily managed his estate, "Needwood", which he farmed mostly using enslaved labor, as had his father (who died in 1819).
[9] In 1843, his son, named like this man's father Thomas Sim Lee, built a mansion at Needwood, now in Burkittsville, Maryland, that is eligible for inclusion on the National Register for Historic Places.
He served as chairman of the committee of the House of Representatives appointed to escort the Marquis de Lafayette from Frederick City to Washington in 1825.
He died on May 17, 1871, while on a visit to his son in New York City, and is interred in New Cathedral Cemetery, familiarly called "Bonnie Brae," in Baltimore, Maryland.