The family were said to be related to the Duke of Portland through his mother, Henrietta daughter of General John Scott of Fife, but the connection has yet to be established.

[3] Lillie's sister, Alicia, married as his second wife, Hugh Mill Bunbury of Guyana; their daughter became a noted Carmelite nun, while their youngest son, Charles Thomas fought in the Crimean War and was promoted Colonel.

Andrew Sutherland RN, Commissioner of Gibraltar and his wife Louisa Colebrooke, on 22 January 1820 at St George's, Hanover Square, Middlesex.

He joined the Lusitanian Legion of the Portuguese Army in the rank of captain, when still only 19 years old, under the watchful eye of Lieutenant-General William Carr Beresford, who raised the famous élite corps of light infantry, the Caçadores.

In 1822, he had bought 'the Hermitage', a grand villa once owned by the dramatist, Samuel Foote, with a fourteen-acre estate at North End, in the Parish of Fulham in the County of Middlesex.

Lillie, who was already a shareholder in the Hammersmith Bridge project,[17] next became a major investor and actor in the Kensington Canal company, a scheme eventually bitterly opposed by Lord Holland.

[19] Around 1830, he also built the 'North End Brewery' and tavern to the South of the highway, together with a maltings to the north on the shorter stretch, initially managed by a Miss Goslin.

[21] Sir John Lillie decided to leave North End in 1837 and moved with his family to Chelsea in the County of Middlesex, where he occupied 12, Cheyne Walk, a 'noble' Georgian mansion, to which he made major structural additions.

During his lifetime, Sir John Scott Lillie took out upwards of 30 patents for all manner of improvements ranging from mechanisms for kneading dough, tilling fields, the chemical composition of stucco to propulsion engines on land and in the water.

[24] Aware of the challenges for transport caused by the transformation of open fields into industrialised areas, Lillie applied himself to the creation of durable street paving.

[29] In January 1835 Sir John Lillie stood for election as a burgess in the borough (constituency) of King's Lynn in the County of Norfolk, but despite loud support, he only managed to come third.

[31][32] In his role as magistrate, Sir John intervened in 1840 in the matter of 'non-restraint of lunatics' at Hanwell Asylum, where it was reported that a lax approach was leading to self-harm by some patients.

[36] He himself died at his residence in Norfolk Terrace, Kensington on 29 June 1868 and was buried at Brompton Cemetery, very close to his earlier efforts on behalf of transport development in Fulham.

Other craft were quickly despatched to rescue the luckless victims and Lillie, being a strong swimmer, assisted a nine-year-old boy who had been with the party and could not swim.