In 1779 he decided to take holy orders, and was appointed to the living of Wrington in Somerset, the birthplace of John Locke and the abode of Hannah More, at whose house he was a frequent and welcome visitor.
Another son, Henry Daniel, was in holy orders, and was chiefly instrumental in the erection of the English church at Athens.
Lady Anne had originally written her words to a Scottish melody previously known as 'The Bridegroom greets' but Leeves's music at once superseded the old tune.
published in 1843, Leeves brought out about 1790, in conjunction with Henry Harington of Bath and Mr Broderip of Wells, a volume of glees.
In 1812 he published 'Six Sacred Airs, intended as a Domestic Sunday Evening Recreation, accompanied by a Pianoforte or Harpsichord, two of them by a Violoncello Obligato or Violin.'