He was educated at Repton School, matriculated from Pembroke College, Oxford, on 28 October 1779, and graduated B.A.
Entering holy orders, he was appointed by his kinsman, John Corbet of Sundorne, in 1786 to the perpetual curacies of Battlefield and Uffington in Shropshire; and on 13 June 1817 All Souls' College presented him to the rectory of Chelsfield in Kent, all of which livings he held until his death.
At an early age Williams became interested in the study of antiquities and topography; and, though he did not print any works, he left behind him a great many manuscripts on the history and antiquities of Shropshire, and executed beautiful drawings of all the parish churches, the principal gentlemen's seats, and the monuments in the county.
Williams gave considerable assistance to John Brickdale Blakeway in his ‘Sheriffs of Shropshire’ and ‘History of Shrewsbury,’ and to Archdeacon Joseph Plymley in his ‘Agricultural Survey of Shropshire.’ During the latter years of his life Williams discontinued his antiquarian pursuits, and devoted himself entirely to his parochial duties.
Seven volumes of his manuscripts, which passed from Lord Berwick's possession to that of Sir Thomas Phillipps, were purchased at Sir Thomas Phillipps's sale on 20 May 1897 for the Shrewsbury Free Library; these included a transcript of the cartulary of Haughmond Abbey, with an index of names and places; historical, topographical, and genealogical collections relating to Shropshire (4 volumes folio); and collections (2 volumes folio) for the ‘History of Shropshire.’ Other volumes of Williams's manuscripts were: a transcript of the cartulary of Shrewsbury Abbey, with an index of names and places; transcripts from 154 Shropshire parish registers; a volume of monumental inscriptions, notes of effigies, and extracts from records; and a list of Shropshire plants.