[1] His father was presumably a British army officer posted to the Bermuda Garrison, possibly Lieutenant Hugh Stewart[2] of the detachment of invalid regular soldiers belonging to the Royal Garrison Battalion, which was disbanded in 1784, following the Treaty of Paris, probably resulting in Stuart's emigration from the colony; the surviving parish registries for the period, compiled by AC Hollis-Hallett as Early Bermuda Records, 1619-1826, list no birth of a Stuart, Stewart, or Steward in or about 1783 other than an unnamed child of Lieutenant Steward, baptised in St. George's on 8 December 1781.

By 1821, he was involved with the black refugees (fugitive slaves) who were beginning to arrive in the area from south of the border.

He began a small black colony near Amherstburg, where he actively assisted the new arrivals to start new lives as farmers.

There he met the young Theodore Dwight Weld, who became one of the leaders of the American abolitionist movement during its formative years.

One hundred and thirty of the more notable delegates were included in a large commemorative painting by Benjamin Haydon.