Pitt's coalition with the Portland whigs in July 1794 and Earl FitzWilliam's consequent appointment as Lord Lieutenant of Ireland gave Ponsonby and his allies an opportunity to regain office.

In 1795, however, he appears to have persuaded FitzWilliam to dismiss John Beresford from his post as first commissioner of the revenue on the grounds of alleged corruption, apparently in revenge for earlier political dealings.

He became then part of the Foxite Whig opposition in the Westminster House of Commons, voting against the Addington and Pitt ministries and in favour of the Prince of Wales and Catholic Emancipation.

By the time Fox regained office in 1806 as member of Grenville's Ministry of All the Talents, Ponsonby's health was poor, with the result that his wife urgently pressed his claims for a peerage, arguing that it was merited by his opposition to the Regency Bill and the Union, and by his staunch support for the Foxite whigs at Westminster.

[citation needed] At a personal level Edmund Burke described Ponsonby in a letter to Lord Charlemont as "a manly, decided character, with ... a clear and vigorous understanding."

He was as interested in sport as he was in politics and was said to have kept 'the best hunting establishment in Ireland' at Bishopscourt, his seat in County Kildare, where it was also reported that he lived 'in the most hospitable and princely style' (Cokayne, The Complete Peerage).