The abbey was founded in 1147 as a daughter house of Clairvaux by Robert, Earl of Gloucester, and was dedicated to the Blessed Virgin Mary.

Early Christian crosses found in the close vicinity and conserved in the nearby Margam Stones Museum suggest the existence of an earlier Celtic monastic community.

The third abbot, Conan, enjoyed the praise of Giraldus Cambrensis, whom he appears to have entertained prior to his official visit with Baldwin of Forde, Archbishop of Canterbury, to preach the Crusade in 1188.

[4] The abbey was dissolved by King Henry VIII of England in 1536 and sold to Sir Rice Mansel.

The purpose of this building is thought to have been to allow members of the monastic community who were engaged in the keeping of flocks to fulfil their devotional obligations without having to return to the main church.