The walls had casemates and a stone corner bastion, which echoes the Italian style of fortification.
During the Thirty Years' War, the succession dispute between the Bishopric of Paderborn and the Counts of Waldeck revived.
For the Bishopric, troops under General Pappenheim, besieged the Pyrmont fortress in 1629, and its 400-man garrison surrendered after ten months.
Rebuilt in haste, it served as a summer residence for the counts of Waldeck, but it was also neglected in the period that followed and began to deteriorate.
When Count Anthony Ulrich of Waldeck-Pyrmont took over the regency of Waldeck and Pyrmont in 1706, master builder, Hermann Korb, built a new schloss in the Baroque style.
In 1956, the state of Lower Saxony acquired the fortress and castle from the princely family of Waldeck and Pyrmont.
From 1984 to 1987, architect Karl-Heinz Lorey renovated and redesigned the schloss and fortifications for use by the then district folk high school and as a museum.