He was the son of the former Susanne Henriette de Boisragon and Dr. Daniel Peter Layard, the physician to Princess Augusta of Saxe-Gotha (the wife of Frederick, Prince of Wales and mother of King George III).

[3] His paternal grandparents were Major Peter Layard and Marie Anne Crozé.

He was the Minister at the Oxenden Chapel, a daughter church of St Martin-in-the-Fields before holding incumbencies in Wootton Bassett, Uffington and Kewstoke.

[3] Through his daughter Charlotte's first marriage, he was posthumously a grandfather of three: Lady Charlotte Guest, George Bertie, 10th Earl of Lindsey, and Montague Bertie, 11th Earl of Lindsey.

[14] Sir Charles was the father of Charles Peter Layard (1849–1916), the 18th Chief Justice of Ceylon, and the grandfather of Sir Herbert Layard Dowbiggin (1880–1966), the British colonial Inspector General of Police of Ceylon.