It was built around a statue of Padre Jesús Nazareno (Father Jesus of Nazareth), which was originally sculpted in 17th century Seville, and had been sent to a Spanish fortress in Morocco.
The icon created great devotion, and in 1689, the 8th Duke of Medinaceli erected a chapel for the statue.
In 1710, a lay fraternity was founded called the Congregation of Slaves of Jesús Nazareno.
In 1890, when the convent of the Capuchins was being dismantled, the mother of the 17th Duke of Medinaceli, Doña Casilda Salabert y Arteaga, the 9th Marchioness of Torrecilla, decided to erect a church.
In 1928, the group was elevated to a primary archconfraternity by the Pope Pius XI.