[1] James was the son of the former Georgina Isabella d'Aguilar and Vice Admiral The Hon.

Following his father's death, his mother married secondly, in 1797, Lt.-Col. Richard Fitzgerald, who was killed in action at the Battle of Waterloo.

[1] He left the House of Commons to become British governor of Ceylon from March 1837 to 1840, and then Lord High Commissioner of the Ionian Islands (based in Corfu) from December 1840 to 1843.

[5][6] In 1819, Stewart-Mackenzie sold the house and grounds of Woodfall Gardens, Glasserton to Stair Hathorn-Stewart at the neighboring Physgill estate.

[12] Through his eldest son Keith, he was a grandfather of James Alexander Francis Humberston Mackenzie, who was created Baron Seaforth in 1921, and of Mary Jeune, Baroness St Helier, society hostess and politician.