Sidney was the only son of Sir John Shelley-Sidney, 1st Baronet, and Henrietta Hunloke.

On 13 August 1825, he married Lady Sophia FitzClarence, illegitimate daughter of King William IV and his mistress, the actress Dorothea Jordan.

Lord and Lady De L'Isle and Dudley had four children: Sidney represented Eye in the House of Commons from 1829 to 1831 and also served as an equerry to his father-in-law from 1830 to 1835 and as Surveyor-General of the Duchy of Cornwall from 1833 to 1849.

In 1835, fourteen years before he succeeded his father, he was raised to the peerage as Baron De L'Isle and Dudley, of Penshurst in the County of Kent.

[1] Lord de L'Isle and Dudley died in March 1851, aged 50, and was succeeded in his titles by his son Philip.