He later went to Salem where he entered the private school of Abiel Chandler and John Brazer Davis, where he prepared for college.

[1] After leaving Harvard, Lee studied law under the direction of John Pickering, Esq.

[2][3] Lee's son George joined the firm in November 1848 and was admitted as a partner on April 1, 1853.

Morgan & Co.[1] On July 18, 1826, Lee was married to Harriet Paine Rose (1804–1885), who had been born on Antigua Island in the British West Indies.

[1] Together, they were the parents of ten children, nine of whom survived to adulthood, including:[1] Lee died in Salem, Massachusetts, on November 19, 1877, after spending the summer with his family in North Conway, New Hampshire.

[6] Through his daughter Marianne, he was a grandfather of Endicott Peabody (1857–1944), an Episcopal priest who founded the Groton School for Boys.

[9] Through his daughter Rose, he was a grandfather of attorney Endicott Peabody Saltonstall (1879–1922) and a great-grandfather of Massachusetts Governor and U.S.