In 1861 the architect Heinrich Freiherr von Ferstel was commissioned to design a monument to Karl Philipp, Prince of Schwarzenberg on Vienna's Glacis and a square named Schwarzenbergplatz.
It was built for Emperor Franz Joseph I's younger brother Archduke Ludwig Viktor, but it was only completed in 1866 and so he did not use it long before being banished to Schloss Klessheim in Salzburg.
After a major renovation, Franz Joseph I handed the building over to the Militärcasinoverein for "his officers forever".
The palace is modeled on the Italian Renaissance.The main facade is oriented towards Schwarzenbergplatz and is dominated by a broad central projection.
The coat of arms of Archduke Ludwig Viktor, flanked by caryatids and a triangular gable, completes the central axis.