At the age of 18 on December 30, 1820, he was appointed midshipman in the United States Navy and 8 years later promoted to lieutenant on May 17, 1828.

When Union forces regained it, he was put in charge of batteries at Drewry's Bluffs, Virginia.

On May 6, 1864, he became chief of the Confederate Navy's Bureau of Orders and Detail, replacing Captain John K. Mitchell.

As late as 1863 he denounced South Carolina for "getting us into this snarl" of secession, complaining that Robert and his family had persuaded him to act against his love of the U.S.

After the war Lee became a farmer and died at his residence at Richland in Stafford County, Virginia on July 22, 1869.