Jones was born on 18 November 1755 in Abergavenny, south Wales and studied at Jesus College, Oxford, from 1773 or 1774 until 1777.
He became friends with Thomas Charles, who was studying at the college at the same time, and corresponded frequently with him thereafter.
He broke off his studies in 1778 on being appointed tutor to the family of a Government servant in Jamaica, but returned to England in 1780, graduated and was ordained.
Twelve years before his death, he acquired a coffin for use when he died.
However, when he died in Broxbourne on 12 October 1821, the coffin proved to be too small for him.