He was the fifth but eldest surviving son of The 10th Earl of Wemyss and his wife, Lady Anne Frederica Anson.

His sister, Evelyn Charteris, was married to John Vesey, 4th Viscount de Vesci; their only child (Mary Gertrude Vesey) was the second wife of Aubrey Herbert (second son of The 4th Earl of Carnarvon), whose daughter Laura Herbert married the writer Evelyn Waugh and was the mother of Auberon Waugh.

He returned to the House of Commons in a by-election in 1886 as one of two representatives for Ipswich and held it in the subsequent general election and would continue to hold it until 1895.

[4] Among their children were:[2] By 1912, Lady Angela Forbes was his mistress, sharing his house in East Lothian, while his wife lived at Stanway in Gloucestershire.

[7][8] Although he and his mistress lived together for many years, he remained married, and his wife became Countess of Wemyss when he inherited the earldom.