Thomas Anson was born 14 February 1767, the first son of George Anson, of Shugborough and Mary Vernon, the daughter of George Venables Vernon.

[1] The family's surname was Adams until 1773, when his father inherited the Anson estates upon the death of his maternal uncles and his unmarried older brother, Thomas.

[2] He was the brother of Mary Anson (born 1759), who married Sir Francis Ford, 1st Baronet, in 1785.

On leaving the House of Commons on 17 February 1806, Anson was created Viscount Anson,[1][4] of Shugborough and Orgrave, Co. Stafford, and Baron Soberton, of Soberton, Co. Southampton, in the Peerage of the United Kingdom.

[4] On 15 September 1794, Anson married Lady Anne Coke, the daughter of the 1st Earl of Leicester of Holkham.