[1] The original central tower of Pfalzgrafenstein Castle was built in 1326/27 by King Ludwig (Louis IV) of Bavaria to increase his toll revenue.

In reaction, Pope John XXII incited his bishops of Mainz, Trier and Cologne against Ludwig as they were concerned his construction threatened to undermine their nearby tolls being collected at the Mouse Tower in Bingen and at Stolzenfels Castle, near Koblenz.

[5] In c. 1340, it was likely due to this dispute with the church that caused Ludwig to build a six-sided turreted wall around the tower, 12 metres high, to improve its defence and to serve as a better ice breaker.

[2][4] Pope John XXII thrice requested his bishops of Mainz, Trier and Cologne to take action against the toll at Kaub, but nothing was done.

The final male descendent of the Counts of Katzenelnbogen, Philipp I, dies in 1479, which causes a 44 year-long inheritance dispute lasting until 1523.

[10][dubious – discuss] In 1504, the castle withstood a 39 day siege by Landgrave William II of Hesse in the War of the Succession of Landshut.

[8][11] Additions to Pfalzgrafenstein Castle were made in 1607 by Elector Friedrich IV, adding the gun bastion pointing upstream, interior arcades and completing the outer wall as it's seen today.

[17] The island of the castle was used for the crossing of the Rhine by Prussian and Russian forces under Field-Marshal von Blücher in pursuit of Napoleon, following the decisive French defeat at the Battle of Leipzig.