In 1800, he was ordained to the curacy of Ravenstone in the diocese of Lichfield and Coventry, and in the course of a ministry of about sixty years was successively curate of Ripple, in the diocese of Worcester; Ross in that of Hereford; lecturer of St. Martin's, with the chapelry of St. Bartholomew's, Birmingham; perpetual curate of Waterfall in Staffordshire on 7 Sept. 1801; minor canon of the cathedral of Worcester, with the rectory of St. Clement's in that city on 5 February 1811; rector of Tretire (he rebuilt the church at his own cost in 1857), with Michael-church, in the gift of Guy's Hospital, on 17 Jan. 1812; minor canon of the cathedral of Gloucester; and vicar of St. John's, Cardiff, in the gift of the dean and chapter of Gloucester on 10 Jan. 1822, which he held with Tretire till the Christmas of 1863.
Webb was a devoted student of antiquities (he was elected a fellow of the Society of Antiquaries in 1819), learned in Latin and in Norman-French, and was skilful in palæography.
Mehul's oratorio ‘Joseph’ and part of Haydn's ‘Seasons’ were adapted by him for the Birmingham musical festival.
He wrote the words for the oratorio ‘David,’ first performed in 1834 at the Birmingham musical festival (1834, 4to), composed by his intimate friend Chevalier Newkomm, which was received in America with enthusiasm, and he prepared a similar foundation for a libretto of Mendelssohn's projected but unaccomplished oratorio, ‘The Hebrew Mother.’ Webb died at Hardwick Vicarage, his son's residence, 18 February 1869, being buried at Hardwick.
He married Sarah (1776–1849), niece of Judd Harding of Solihull in Warwickshire, whose family traced descent to Shakespeare's kindred, and had by her Thomas William Webb, and a daughter Frances, who died in infancy.
He left unfinished an edition for the Camden Society of the manuscript ‘Military Memoirs of Colonel John Birch,’ which was published in 1873, and ‘Memorials of the Civil War as it affected Herefordshire,’ which was published in 1879 by his son Thomas William Webb (London, 2 vols.