Henry VI of Pyrmont had the Reichsacht imposed on him as a result of the inheritance dispute and the administration of his share of the castle was transferred to his brother Frederick.
[citation needed] The castle did not witness more peaceful times until the second half of the 15th century, when Emperor Maximilian I elevated Henry IV, Lord of Pyrmont, to the status of a Freiherr.
In 1652, one of the Eltz heiresses sold her share to members of the family of Waldbott of Bassenheim because of the ongoing disputes who, two years later, were appointed imperial Freiherren thanks to their ownership of this estate.
In 1695, another Eltz share in Pyrmont Castle went to the Electorate of Trier and was also acquired by the Waldbott of Bassenheim family in 1710.
She suffered the fate of many castle estates west of the Rhine: in 1810, she was auctioned off by the French, with seven hectares of land, for 4,550 francs.
[citation needed] In 1818, Count Friedrich Waldbott von Bassenheim bought back the castle.
Only the family of the architect, Franz Krause, who worked as a draughtsman for The Art Monuments of the Rhine Province, made part of the dilapidated castle complex habitable again from 1912 onwards.
In 1963, two Düsseldorf architects, Helmut Hentrich and Hubert Petschnigg, bought the remnants of Pyrmont Castle.
The inner bailey, built on the rocks high above the Zwinger, consists of the formerly three-storeyed palas, the attached cookhouse and the bergfried.