He sits as a Crossbench member of the House of Lords, and previously served as Ambassador to France and Permanent Under Secretary at the Foreign and Commonwealth Office.

He was appointed Director General for European and Economic Affairs 1994–96, following which he became a Senior Associate Member of St Antony's College, Oxford.

In July 2001, he was appointed Permanent Under-Secretary of State for Foreign Affairs and thus Head of the Diplomatic Service, a post he took up on 14 January 2002.

In 2005 and 2006, Jay served as the Prime Minister's Personal Representative (Sherpa) for the G8 summits at Gleneagles and St Petersburg in addition to his PUS duties.

He has been a non-executive director of Associated British Foods (2006-), Credit Agricole (2007-2011), EDF (2009-), Candover PLC (2008-)and Valeo SA (2007-).