His father was involved in the family counting house business and was a failed East India merchant.
Lee read Mathematics at St John's College, Cambridge between 1802 and 1806, graduating fifth wrangler in his year.
[1] In 1833 Lee married Cecilia Rutter (23 July 1782 – 1 April 1854), of Lymington, whom he described as of "humble station but excellent character and good disposition".
The properties included Colworth House near Sharnbrook in Bedfordshire and Totteridge Park, Buckinghamshire (formerly in Hertfordshire).
[1] Lee would "invite thirty or forty scientists to stay with him for anything up to a month, discussing their latest theories" "year after year"; these regular guests included William Henry Smyth and his daughter Henrietta, mother of Scouting founder Robert Baden-Powell, 1st Baron Baden-Powell.
His scholarly nature meant Lee cared nothing for "the aristocratic world which his house so vividly evoked" for visitors.