He was the son of William Bruce, 8th Earl of Kincardine and Janet Roberton.
He was a founding member of the Royal and Ancient Golf Club at St Andrews.
He is buried in the southern transept of Dunfermline Abbey close to the grave of Robert the Bruce.
In 1812, Scottish composer Magdalene Stirling named her Charles Bruce Reel after him.
[4]On 1 June 1759, Elgin married Martha Whyte (1739–1810), who later became governess to Princess Charlotte of Wales.