He was then elected MP for Buckinghamshire, succeeding William Selby Lowndes, and serving with the Marquess of Chandos.

After his father's death in 1838, and on his inheritance of the barony, he was succeeded on the Wycombe seat by his first cousin, George Robert Smith.

Charlotte Augusta Annabella Drummond-Willoughby (1815–1879), daughter of Peter Drummond-Burrell, 22nd Baron Willoughby de Eresby, and Lady Sarah Clementina Drummond.

Harry Legge-Bourke, MP for Isle of Ely 1945–1973, was his great-grandson through his first son Charles.

Lord Carrington was also a descendant in the Colville line; his father, the 5th Baron, married the Hon.