He was of Buckinghamshire origin, the younger son of the Reverend William Cleaver, who ran a school at Twyford, and his wife Martha Lettice Lushden.

He was briefly Bishop of Cork and Ross, before in 1789 being translated to Ferns and Leighlin.

[1] During the 1798 insurrection in Ireland his palace in Ferns was ransacked and Cleaver was obliged to take refuge in Beaumaris, Anglesey which was in his brother, William Cleaver's diocese of Bangor, Gwynedd, where he lived at what is now the Bishopsgate Hotel.

[2][1] His exercise of the Archbishopric of Dublin was cut short for reasons of alleged insanity.

He married Catherine Wynne of Hazelwood, County Sligo, by whom he had several children, including William, Frances and Caroline; Caroline married Admiral James William King, and was the mother of the prominent evangelist Catherine Pennefather.