The Pénitents bleus called on Toulouse's finest artists to decorate their chapel, which didn't suffer too much during the French Revolution.
The first meeting of the fraternity was held on 29 September 1575 in the chapel of the Collège Saint-Martial in Toulouse (where the Grand Hôtel de l'Opéra now stands).
A ruling by the King's Council on 11 September 1620 obliged the Blue Penitents to return the building to the previous owners, which they did in 1626, after having had the Saint-Jérôme church built.
The foundation stone was laid by the Bishop of Pamiers, Mgr d'Esparbès de Lussan, on behalf of King Louis XIII.
When the chapel became a parish church in 1805, Toulouse city architect Jacques-Pascal Virebent was commissioned to adapt the building to its new function.
They are linked to the church by a gallery known as the "Passage Saint-Jérôme", which features a number of showcases displaying ecclesiastical works of art.