Parry was born on 25 November 1754 at Abergavenny, Monmouthshire, where his father was a deacon of the Baptist congregation.
On the advice of Samuel Brewer, Parry entered Homerton Academy on 8 February 1774, as a candidate for the Congregational ministry.
[1] In 1780 Parry finished his course, left Homerton, and was ordained to the ministry at Little Baddow in Essex.
[1] Parry published three letters to Lord Aylesford, chairman of a meeting held at Warwick on 2 February 1790 to oppose the removal of the dissenters' disabilities.
His works include:[1] Seventeen volumes of Parry's lectures in manuscript went to the Historical Library at New College London.