The structure originated as a 13th-century fortress tower, belonging to the Ricchieri family, who served at times, either the Holy Roman Emperor or the Republic of Venice.
By the 15th century, the main layout was developed consisting of three blocks with a frescoed facades, and lower porticos marked by arches.
The piano nobile or main salon is a hall frescoed with the Story of Tristan and Isolde from the early 14th century.
Also a series of small painted wooden tablets depict scenes from the age: including romances, duels, hunts, and a bestiary with unicorns, dragons, and flying serpents.
Major portions of the palace were donated in the 1950s by the Ricchieri family to the Comune for the civic museum.