Born in London on 6 October 1793, Barrington was the eldest son of fifteen children born to the Reverend George Barrington, 5th Viscount Barrington, by his wife Elizabeth, second daughter of Robert Adair and Lady Caroline Keppel (the second daughter of Willem van Keppel, 2nd Earl of Albemarle),[1] a descendant of Charles Lennox, 1st Duke of Richmond.

Like his father, he was educated at Westminster School and Christ Church, Oxford, where he graduated with a Bachelor of Arts in 1814.

Constance Mary Barrington (1847–1926), who married Lawrence Palk, 2nd Baron Haldon, Hon.

Through his daughter Adelaide, he was a grandfather of Charles Barrington Balfour (a first cousin of Arthur Balfour, who served as Prime Minister from 1902 to 1905), an army officer and MP for Hornsey who later served as the Lord Lieutenant of Berwickshire.

He married Lady Helena McDonnell, daughter of Alexander MacDonnell, 5th Earl of Antrim.