In 1795 he was sworn of the Privy Council and became Irish chief secretary under Pitt's government, retiring in 1798.

He was Home Secretary from July 1801 to August 1803 under Addington, who made him Chancellor of the Duchy of Lancaster in 1803.

[3] Lord Chichester married Lady Mary Henrietta Juliana, daughter of Francis Osborne, 5th Duke of Leeds, in 1801.

Lord Chichester died in July 1826, aged 70, and was succeeded in his titles by his eldest son, Henry.

His daughter, Lady Lucy Anne Pelham, married Sir David Dundas.