Clifton was born on 3 March 1845 into a prestigious Lancashire family.
[2] He was the only son of John Talbot Clifton and the Lady Eleanor Cecily Lowther.
[3] His father, who succeeded Thomas' grandfather to the Lytham Hall and estate in 1851, was a Member of Parliament for North Lancashire between 1844 and 1847 and died in Algeria in 1882, two years after his death.
[3] On 7 February 1867, he married Madeline Agnew at St George's, Hanover Square in London.
Madeline was one of thirteen children born to the former Lady Mary Arabella Louisa Noel (a daughter of Charles Noel, 1st Earl of Gainsborough) and Sir Andrew Agnew, 8th Baronet, MP for Wigtownshire.