[2] Together the couple had one child, James George Alexander Bannerman Carnegie, 3rd Duke of Fife, 12th Earl of Southesk (23 September 1929 – 22 June 2015).
Asked about his membership in later life, Southesk professed ignorance of all the aims of the club but praised Ramsay as "a very loyal, patriotic man".
Lord Southesk later married Evelyn Julia Williams-Freeman (27 July 1909 – 30 August 1992) on 16 May 1952 at Scone Palace in Perthshire, Scotland.
Evelyn, Lady Southesk died aged 83 at Kinnaird Castle, Brechin in Angus, Scotland six months after the death of her husband.
Lord Southesk died on 16 February 1992 at age 98 at his home Kinnaird Castle, Brechin in Angus, Scotland after a brief illness of influenza, having spent the last year of his life as an additional vice-president of the Conservative Monday Club, of which he had long been a member.