He is the ancestor of the Stewarts of Ardvorlich on Lochearnside, whose family history is recounted by Sir Walter Scott in A Legend of Montrose.

James the Fat would never return to Scotland, and he was unable to inherit the Albany estates, which were forfeited following his father's trial and execution.

However, unlike his father, James "Beag" Stewart did not assert this claim and he was eventually able to secure a royal pardon and return to Scotland, becoming a loyal servant of the crown.

He is the ancestor of the Stewarts of Ardvorlich on Lochearnside, whose family history is recounted by Sir Walter Scott in A Legend of Montrose.

[5] Alexander Campbell, writing in 1812, tells the following story: James Stewart married Annabel Buchanan, daughter of Patrick, 14th Laird of Buchanan,[1] and was granted the estate of Baldorran, (aka "Balindoran"), Stirlingshire, by his cousin John Stewart of Damby in 1457.