His middle name (Bertie) was from his mother's family, the Earls of Abingdon, descended from a Tudor courtier who married the Dowager Duchess of Suffolk (herself suo jure Baroness Willoughby de Eresby).

Guest was educated at Harrow School in Middlesex, and he went on to gain a Master of Arts degree in 1856 from Trinity College, Cambridge.

She was the daughter of John Spencer-Churchill, 7th Duke of Marlborough, thus making Guest an uncle-by-marriage of Winston Churchill, later the Prime Minister of the United Kingdom.

Their children included the following: His wife Cornelia, helped oversee the philanthropic project of building estate cottages to improve the living conditions of workers in Dorset.

[13] Wimborne acquired the Highland sporting estate of Glencarron (Ross-shire) in the 1860s, building a Lodge (Alexander Ross / William Joass, c. 1868[14]).