He is known for being the maternal great-grandfather of Pitt the elder, prime minister of Great Britain from 1766 to 1768.

He was also the uncle of two royal mistresses, Barbara Villiers (mistress of Charles II of England), and Elizabeth Hamilton, Countess of Orkney (presumed mistress of William III of England).

His son, Edward FitzGerald-Villiers died in 1693, so the title went to his grandson, John Villiers, 1st Earl Grandison.

In 1662, He was appointed Captain of the Yeomen of the Guard, and remained in that position until 1689, when he was removed from it by King William III.

As captain, he was one of the most senior figures in the Royal household, which resulted in him walking immediately behind James II of England, during his coronation procession in 1685.