Under the Lyndon B. Johnson administration relations were more formal but remained excellent; and Ormsby-Gore maintained his position after the Labour government took power in Britain in 1964.

A fierce opponent of oil-barrel politics, Ormsby-Gore's terse dismissal of the phenomenon ran: "It would indeed be a tragedy if the history of the human race proved to be nothing more than the story of an ape playing with a box of matches on a petrol dump."

Unable to persuade the American government to agree with the British line over Yemen and the Congo, or to proceed with either a negotiated settlement with Soviet premier Nikita Khrushchev over Berlin or the Skybolt ballistic missile programme, he nevertheless played a significant role in the Cuban Missile Crisis and ensured that Britain's views were taken into account by the American government.

They persuaded him to act like a statesman and conclude Test-ban treaties with Russia and not fear being branded as an appeaser by political opponents in the United States.

He encouraged Kennedy to remain focused on issues relevant to the world and the future, rather than attempting to protect himself politically.

According to the Duchess of Devonshire, who travelled with the British delegation to Kennedy's funeral in November 1963, Macmillan's successor as Prime Minister Alec Douglas-Home had wanted to appoint Ormsby-Gore as Foreign Secretary, but R. A. Butler had insisted on having this post as a condition of serving under Home.

[10] In 1968, when she married the Greek shipping tycoon Aristotle Onassis, Lord Harlech was opposed and wrote to her asking her to change her mind.

He also had a successful career as a television executive, founding HTV, and served as president of the British Board of Film Classification.

He had an active interest in the avant-garde, and for nearly ten years, beginning in 1969, was patron of the Institute for Research in Art and Technology.

[7] Jacqueline Kennedy declined his offer of marriage in a letter, writing: "If ever I can find some healing and some comfort — it has to be with somebody who is not part of all my world of past and pain ...

Contents of Lord Harlech's house at Glyn Cywarch, including his government despatch box containing drafts from him and numerous letters from Jackie Kennedy illustrating their close relationship, were auctioned at Bonhams in London in September 2017 on behalf of his grandson after a preview in New York.