In 1799 he was sent out on another special military mission, this time to the headquarters of the Austrian commander, Archduke Charles, to attempt to persuade him to retain his troops in Switzerland rather than removing them to the Middle Rhine, but he was unsuccessful.

In 1794 he was granted a British peerage as Baron Mulgrave,[1] entering the House of Lords, and in 1796 he was made Governor of Scarborough Castle.

Following an accident suffered by Lord Harrowby, Mulgrave took his place as Foreign Secretary, in which position he helped Pitt to form the Third Coalition against Napoleon.

Thomas Grenville, writing to the Marquis of Buckingham, expressed an opinion that he was only "put in ad interim until Lord Wellesley's arrival, who is expected in June".

On 28 January 1806 Mulgrave laid before the House of Lords copies of the treaties recently concluded with Russia and Sweden, to which Prussia and Austria had acceded, and on 4 February he explained their object.

Lord Mulgrave married Martha Sophia, daughter of pottery manufacturer Christopher Thomson Maling, at St Michael's, Houghton-le-Spring in 1795.

He died in April 1831, aged 76, and was succeeded by his eldest son, Constantine, who was later created Marquess of Normanby.