He was born on 3 April 1756, at Stonehaven, Kincardineshire, he was educated at Marischal College, Aberdeen, was ordained deacon in the Scottish episcopal church on 1 July 1776, and admitted priest on 19 March 1777.
Jolly continued to discharge at the same time the duties of an ordinary pastor in Fraserburgh, where he lived by himself in a plain two-story house in Cross Street.
He read daily a fixed number of pages of the Hebrew bible and the Greek New Testament, and portions of the Church Fathers, especially John Chrysostom and Augustine of Hippo.
During 1826 he published a short treatise entitled ‘A Friendly Address to the Episcopalians of Scotland on Baptismal Regeneration,’ a reply to the attacks made on Scottish episcopal teaching by the Rev.
[1] Jolly's most popular work was ‘Observations upon the several Sunday Services and principal Holydays prescribed by the Liturgy throughout the Year,’ 1828; 3rd edit., Edinburgh, 1840, with memoir by James Walker, bishop and primus.