In 1463 Antonio Todeschini Piccolomini completed the building as seen today, with an open gallery in the second floor of the courtyard and improvements in the walls around the castle.

The following families succeeded in the castle (like Sforza-Cesarini and Dragonetti) didn't change further the structure of the building.

A deep restoration took place since 1940, put on hold during World War II, restarted in 1955 and completed in 1960.

Now the building is used as offices for the Ministry of Cultural Heritage and Activities, in addition to a museum of sacred arts.

The building is protected by irregular walls with several towers, part of them with a squared plan, and other ones trunk-conical shaped.