Lady Anson was born at Holkham Hall[3] on 25 January 1779[4]: 65 [5]: 233 [b] to Thomas Coke (who in 1834 was raised to the peerage as Earl of Leicester) and Jane Dutton.

He was a member of parliament, worth £22,000 (equivalent to £3,192,563 in 2023) per year, and heir to Shugborough estate in Staffordshire.

[4]: 67  The Duke of Sussex said that he was a "true manly, noble, splendid fellow, possessing much of the real English character, sound sense, and although perhaps hurried away a little too much by country sports, has a great deal of good in him.

"[4]: 67–68 According to Susanna Wade Martins, Anne was described as "thin, excitable, energetic, never quiet, constantly getting into quarrels, but always ready to help others.

[4]: 67  Mrs. Powys, who visited Shugborough in 1800, stated that competent works by Anson were in every room of the house, including three full-length paintings of her children.