The priory was founded in 1268 by Thomas FitzMaurice FitzGerald, the second baron of Desmond, whose maternal grandfather, Maurice FitzGerald, 2nd Lord of Offaly, had founded the Franciscan Friary of South Abbey, Youghal.

The Dominican priory was initially dedicated to the Holy Cross, but was changed to 'Our Lady of Graces' in the late 15th century following the re-discovery of a small, ivory statue of the Madonna and Child.

Samuel Lewis in his Topographical Dictionary of Ireland, written in 1837, said of it: "Thomas, whose son, in 1263 or 1271, founded a Dominican monastery, called the Priory of St. Mary of Graces."

The statue can now be found in St Mary's Dominican Priory in Cork City.

Lewis goes on to say in his book that: The western gable and some of the eastern portions of the Dominican friary, at the north end of the town, still remain ... Near the south end of the town is a chapel of ease, a neat plain building, erected in 1817 on the cemetery of the ancient Dominican friary, at an expense of £1200, of which £900 was a gift from the late Board of First Fruits and £300 was raised by subscription Some ruins of the priory can still be seen, and the area is also home to the main cemetery of Youghal.