His elder brothers were George Chichester, 2nd Marquess of Donegall, and the Hon.
His father served as MP for Malmesbury from 1768 to 1774 before being elevated as 1st Marquess of Donegall in 1791.
[4] On 8 August 1795, Chichester married Lady Anne Harriet Stewart, daughter of John Stewart, 7th Earl of Galloway, and Anne Dashwood (a daughter of Sir James Dashwood, 2nd Baronet).
Together, they were the parents of two sons and three daughters, including:[4] Lord Spencer died at Paris on 22 February 1819.
His eldest son, Arthur, was raised to the peerage as Baron Templemore on the occasion of the coronation of William IV in 1831.