After the order was banned by an imperial edict in 1782, the land was abandoned until Count Karlo Drašković bought it in 1835 with intention to build a residence for his family.

The only question is whether the palace was built to Felbinger's own plans, because its three blueprints which are kept in the Croatian History Museum are unsigned and undated.

Drašković family never fully settled in their newly built home, so they sold it under favorable terms to the members of the Illyrian movement.

[clarification needed] They intended to use the palace to house the National Museum with a reading room and a casino, as well as the Croatian-Slavonian Economic Society.

The central part of the ground floor housed the Croatian National Museum since 1846, while the wings were occupied by the Husbandry Society, the reading room and the casino.