Williams, from Mallwyd, Merionethshire, Wales, was baptised on 27 March 1779.
After being taught at a school in the local church and by the vicar of Betws-yn-Rhos, he attended Ruthin School and then (from 1798 to 1802) Jesus College, Oxford, obtaining a Bachelor of Arts degree in 1802 and a Master of Arts degree in 1805.
After his ordination in 1802 by John Randolph, Bishop of Oxford, he was an usher at Friars School, Bangor and also curate of Llandygai.
He was married with three sons (including the theologian Rowland Williams) and five daughters.
St Asaph Cathedral contains a memorial window for him at the west end of the building.