It is located in Via del Seminario, between piazza di Sant'Ignazio and the Pantheon in the ancient Campus Martius and in the second sector of the present-day Colonna rione, not far from Via del Corso.

The first palazzo on the site was built by the Gabrielli family, counts of Gubbio, whilst its second name derives from its rebuilding and redecoration by cardinal Vitaliano Borromeo for use by the Jesuits.

In 1774 the Monte di Pietà of Rome purchased the palace for 32,000 scudi.

In 1796 it was given to the fabbrica di San Pietro, however by 1824, it was returned to the Jesuit order.

During the 19th century, the palace ownership changed hands a number of times.