He had offices at Rhayader and lived at Penralley House, Rhayader,[2] He became a noted authority on the archaeology of the Cistercian Monasteries in Wales and undertook excavations at Strata Florida Abbey in Ceredigion, Abbey Cwm Hir in Radnorshire and Strata Marcella near Welshpool in Montgomeryshire.

At this period he continued to undertake railway survey work in Wales and in parts of England[6] In 1864 he was appointed county surveyor for Radnorshire.

He set up the Rhayader Company of the Volunteers, with the rank of Captain and eventually he became Lieutenant Colonel in command of the entire battalion.

Birmingham Corporation had appointed the engineer, James Mansergh, to oversee the project but he needed someone to plan and build the workers' village.

In these matters he was to work closely with James Mansergh, who referred to him as 'our ubiquitous friend'[12] A major scheme in which Williams was involved was the planning and laying out of Llandrindod after the '’Swydd Neithon'’ was enclosed in 1862.

He had fairly diverse interests which included Numismatics, arms and armour, the study of monumental effigies in churches and he contributed an article about Roman finds at Kenchester in Herefordshire.

[18] Shortly before his death in 1899, Williams, as High Sheriff of Radnorshire, was instrumental in holding a groundbreaking treasure trove proceedings on the hoard of late Iron Age or Roman gold bracelets and a ring that were found at Carreg Gwynion, Nantmel near Rhayader.

[20] It was the Cistercian Abbeys of Wales that were to become his main interest and he has been described as the 'father of Cistercian archaeology in Wales'[21] To assist him with his Excavations he employed his architectural assistant, Telfer Smith[22] as clerk of Works and Worthington George Smith to act as an architectural draughtsman.

This led to a more detailed excavation of the nave the Abbey church, sponsored by the Society of Cymmrodorion in 1890, who were hoping to uncover the burial place of Llywelyn ap Gruffudd, Prince of Wales 1258–82.