During the Second World War, Sidney served with the Grenadier Guards in France and Italy; he was awarded the Victoria Cross in 1944 for his actions in the Battle of Anzio.

[2] Later he led another counter-attack and dashed forward, engaging the Germans with his tommy gun at point-blank range, forcing a withdrawal.

His father died in June 1945 and he succeeded as 6th Baron De L'Isle and Dudley, requiring transition to the House of Lords.

During this time he visited Australia, travelling to Woomera to examine weapons research and meeting the Prime Minister, Robert Menzies.

De L'Isle faced no constitutional issues during his time in office; the Official Secretary throughout his term was Murray Tyrrell.

[5] De L'Isle was the first governor-general since William McKell (1947–1953) to have children living at Yarralumla, and this made him popular with the general public.

In 1975 he co-founded what is now called The Freedom Association, a free-market campaign group opposed to the post-war consensus that played a prominent role in the Grunwick Dispute.

Viscount De L'Isle died in Kent on 5 April 1991 and was buried in the Sidney family vault at St John the Baptist, Penshurst.

The couple had five children: After his wife's death, he married the widowed Lady Glanusk (née Margaret Shoubridge) on 24 March 1966 in Paris.

Viscount De L'Isle was a descendant of King William IV though his illegitimate daughter Lady Sophia FitzClarence.