By inheritance and marriage, it passed successively into the hands of the Nesselrode, Nassau, Schwartzenberg families and finally from 1550 to 1812 the property of the Counts of Metternich, except between 1795 and 1798 when the French revolutionary administration confiscated it.

Franz Georg Karl Count of Metternich-Winnenburg-Beilstein sold the family estate in 1812 to a building materials dealer.

The Prussian administration immediately halted the demolition and attempted for the first time to protect the monument.

In the 19th century it was nearly destroyed by quarrying but in 1969 the castle was thoroughly renovated under the impulse of the new owner Professor Jean Overloop.

His wife and daughter, who inherited the castle after his death, donated it to a non-profit organization.