One of his first cousins, John Thomas Foster married Lady Elizabeth Hervey, aka Lady Bess Foster, daughter of Frederick Hervey, 4th Earl of Bristol, later mistress William Cavendish, 5th Duke of Devonshire.
After the separation and divorce from her first husband and the death of her lover's first wife, Lady Georgina Spencer, Elizabth finally married him in 1809 and became the Duchess of Devonshire.
Foster married Catharina-Letitia Leslie (died 23 November 1814) daughter of Rev.
(Leslie, a scion of the family of the Earl of Rothes, was Prebend of Tullycorbet and then of Tandragee.
Peter Leslie, was rector of Ahoghill, and had married Jane, daughter of Most Rev.