Cavendish was the third son of William Cavendish, 4th Duke of Devonshire and the former Lady Charlotte Boyle, daughter of Richard Boyle, 3rd Earl of Burlington of the first creation, whose title had become extinct upon his death in 1753.

[4] Cavendish sat as Member of Parliament for Knaresborough from 1775 to 1780, for Derby from 1780 to 1797 and for Derbyshire from 1797 to 1831.

On 10 September 1831 he was raised to the peerage as Baron Cavendish of Keighley, in the County of York, and Earl of Burlington, a revival of the title held by his maternal grandfather.

With the architect Samuel Ware, he made a number of significant modifications to the house, including the building of Burlington Arcade along the west side.

He died at Burlington House in 1834 and was buried in All Saints' Church, Derby.