The village of Huntley, four miles from Newent in Gloucestershire, had apparently no connection with the Gordon family or with the town of Huntly in North Britain.

The Dictionary of National Biography described him thus: "At the time of his marriage the Duke was reputed one of the handsomest men of his day".

He was an enthusiastic supporter and patron of the music of William Marshall (1748–1833), a Scottish fiddler and composer, and famous for his many strathspeys, who acted as steward of the Gordon household.

However, with his own debts mounting, the Glen Feshie estate was sold to George Macpherson Grant of Ballindalloch in 1815.

Glenbanchor was sold to James Macpherson of Belleville in 1822, the money raised going to fund improvements at Huntly.

[5] Gordon married firstly on 23 October 1767 at Ayton, Scottish Borders, and again at Mr. Fordyce's house in Argyll Street, Edinburgh, Jane, the daughter of the late Sir William Maxwell, 3rd Baronet of Monreith, by his wife, the Lady Magdalen Maxwell, daughter of William Blair, of Blair, Ayrshire.

From 1787 she was part of the social centre of the Tory party and was described in the Female Jockey Club of 1794 as possessing "an open ruddy countenance, quick in repartée, and no one excelling her in performing the honours of the table, her society is generally courted".

Elizabeth Grant mentions "The great width of the Spey, the bridge at Fochabers, and the peep of the towers of Gordon Castle from amongst the cluster of trees that concealed the rest of the building....the Duke lived very disreputably in this solitude, for he was very little noticed, and, I believe, preferred seclusion.

"[7] The Duchess is best remembered for placing the King's shilling between her teeth to help recruitment to the Gordon Highlanders which were founded by her husband.

The Duchess of Gordon died at Pulteney's Hotel, Piccadilly, Middlesex, on 14 April 1812 and was buried at her beloved Kinrara near Aviemore.

The Duke himself died suddenly at Mount Street, Berkeley Square, on 17 June 1827, and was buried in Elgin Cathedral.