In order to avoid Egloffstein Castle being besieged by the Bishop of Bamberg, Götz was excluded from the family association in 1374.

In 1376 the Bishop of Bamberg, Lamprecht of Brunn (1374–1398) forced Seybot II von Egloffstein to grant Öffnungsrecht to the Bishopric in the majority of the castle.

In spite of the constant reinforcement of its defences the castle was conquered by Nuremberg troops during the War of the Succession of Landshut in 1504 and partly razed.

During the conflict between William III of Egloffstein and the Bishop of Bamberg, Veit II of Würtzburg (1561–1577), the latter had the castle occupied and ravaged in 1563.

When the village of Egloffstein was raided by the Swedes in 1632 and 1645 during the Thirty Years' War, the castle suffered severe damage necessitating comprehensive rebuilding work.

In 1750 the brothers, Albert Christopher Charles and Louis I of Egloffstein built the present parish church of St. Bartholomew in place of the lower cabinet and its old castle chapel.