On 2 July 1811, Clifton married Elinor Bell (of Wandle House, Wandsworth, London, a first cousin, close friend and co-religionist of Elizabeth Fry, the famous Quaker prison reformer).

(brother of John Hutt, Governor of Western Australia from 1838 to 1846) and Edward Gibbon Wakefield, upon whose principles of colonisation the company was founded.

Clifton and his family and other first settlers of the Australind region sailed to Australia on the barque Parkfield in October 1840, arriving at Port Leschenault (Bunbury) on 18 March 1841.

Clifton's children became prominent members of society in Bunbury, Australind and Brunswick, several occupying important positions such as Resident Magistrate (Pearce), Inspector of Water Police (George, who later returned to England and became Governor of Dartmoor Prison), member of the Town Trust (Pearce), Collector of Customs at Fremantle (Worsley).

In 1847 Clifton moved to Upton House, which had been built from bricks brought from England as ballast on the Trusty when it arrived in May 1844.

As the country gentleman, he was the personification of hospitality; whilst his agreeable manners, well-stored mind, and hilarity of spirits enhanced in no small degree the pleasures of his way-faring guests.

In his family relations Mr. Clifton was in all respects patriarchal, and although he lived and died 'amidst a grove of his own kindred', there were many absent ones to grieve over his loss.

"[5]Perth Gossip, a play by Canon Alfred Burton,[6] was produced for the West Australian Historical Society in November 1933 at the Karrakatta Club,[7] and again at the Burt Memorial Hall on 8 May 1934.