St. Mary's has been called "the Westminster Abbey of Wales" because of its large size, and the numerous high-status tomb monuments and medieval effigies surviving within it.

[1] Although the Norman building was built around 1070, a previous Iron Age, Roman or Post-Roman structure existed on the site.

[2][3] The current structure was originally the church of the Benedictine Priory, established under Hamelin de Balun the first Norman holder of the title Lord Abergavenny, which in the 1090s became Baron Bergavenny.

Henry de Abergavenny was a prior here and later at Llandaff in the late 12th century and was chosen to assist at the coronation of King John I of England in 1199.

They bear the name of the prior at that time Wynchestre and his own stall remains, slightly raised and surmounted by a mitre.

One effigy is that of John de Hastings, Lord of Abergavenny (died 1324) and shows him as a young knight, wearing a long surcoat over a hauberk and a hood of fine chainmail.

The chapel was dedicated to Saint Joseph, the husband of the Blessed Virgin Mary, by Bishop Richard Pain in November 2017.

The art historian Andrew Graham-Dixon called it the one “unarguably great wooden figure” remaining from the Middle Ages.

[8][9][10] The project was visited in April 2016 by the Archbishop of York, the Most Revd John Sentamu;[11] and the completed work was dedicated in the presence of Charles, Prince of Wales, on 7 July 2016.

Around 1830 a secondhand organ, built by John Byfield in 1760 for the Lord Mayor's Chapel in Bristol,[13] was sold to Mr. H. Smith and moved to the church.

A small 5 stop, one manual chamber pipe organ, originally from St Nicholas' Church, Durham has been in use for small-scale services since 2015.

To commemorate the Golden Jubilee of Queen Victoria in 1887, the bells were rehung and augmented to eight by Llewellins & James of Bristol.

The restored Tithe Barn within the precincts of the Priory, was opened by HRH Charles, Prince of Wales on 23 October 2008 and houses the large 24 feet (7.3 m) tapestry depicting the history of Abergavenny and embroidered by volunteers to mark the 2000 Millennium.