The core of the collection was assembled initially in 1868 from works from suppressed churches and monasteries.
By the 20th century, the local superintendent of art and art historian, Corrado Ricci, added works from his collection and others, for a display in the Palazzo dei Priori.
The collection included works by Domenico Ghirlandaio, Pietro Candido, Priamo della Quercia, Taddeo di Bartolo, and Luca Signorelli, and the famous Volterra Deposition by Rosso Fiorentino.
The Palazzo dei Priori was a cramped locale for the many works, and in 1981 a display of works at the Palazzo Minucci-Solaini induced the authorities to create a new museum at that site, inaugurated in 1982.
This palazzo was built in the late 1400s, with the design traditionally attributed to Antonio da Sangallo the Elder.