[1] He was a junior member of the Clifton family of Nottinghamshire,[citation needed] descended from the 11th century Alvaredus de Clifton, warden of Nottingham Castle "in the time of William Peverell, bastard son of William the Conqueror".

[2] Clifton served as Lieutenant of Dover Castle and as Captain of Pontoise, France where he was knighted.

Clifton's first wife was Isabel Herbert (died c. November 1457), widow of William Scott (died 5 February 1434) of Brabourne, Kent, and daughter of Vincent Herbert (alias Finche) of Netherfield, Sussex by his wife Isabel Cralle, daughter and coheiress of Robert Cralle.

Their two daughters were:[4] Clifton's second marriage, before 20 March 1463, was to Maud Stanhope (died 30 August 1497), widow firstly of Robert Willoughby, 6th Baron Willoughby de Eresby (died 1452), and secondly of Sir Thomas Neville (killed at Battle of Wakefield 30 December 1460), second son of Richard Neville, 5th Earl of Salisbury, executed at Pontefract on 31 December 1460 after the Battle of Wakefield.

Maud Stanhope was the niece and coheiress of Ralph de Cromwell, 3rd Baron Cromwell (d. 4 January 1456), and the heiress of her sister, Joan Stanhope (died 10 March 1490), who had married firstly Sir Humphrey Bourchier (died 1471), slain at the Battle of Barnet on 14 April 1471, and secondly Sir Robert Radfcliffe of Hunstanton, Norfolk.