Before the construction of Schwarzenberg Palace, the site housed several medieval buildings including the workshop of Master Theodoric.

[1][2] It would undergo significant renovations under the Schwarzenbergs, with the dwelling area being fully rebuilt by Anton Erhard Martinelli and Antonín Haffenecker [cs], and the sgaffitos being finished with designs from Josef Schulz and Jan Koula.

[2] The palace became unoccupied over time as members of the Schwarzenberg family moved to Vienna, the capital of the Austrian Empire.

[1][3] The palace its notable for its extensive sgraffito designs, which cover 7,000 square metres (75,000 sq ft) of the buildings surface.

[2][3][5] Most of the sgraffito are repeating black-and-white cuboid blocks, though there are detailed carvings, such as a rooster and owl next to the palace's sundial.