The castle house stands tall, next to the Moselle's right-bank towpath downstream of the strategic Baldwin Bridge (lowest crossing of the river) built in 1342.

Around 1185, on the site of the present castle, the family of von der Arken built a Romanesque residence from the remains of a Roman round tower.

Koblenz's city walls, which at that time still corresponded to those of the Late Roman castellum, were extended in 1250.

The castle was partly built on the ruins of the Late Roman city wall dating to the 4th century.

Between 1898 and 1900, extensive renovation work was carried out under the direction of the city's chief architect Friedrich Wilhelm Ludwin Mäckler.

After the completion of the "Forum Confluentes" in 2013, the library moved into the new cultural building on the Zentralplatz, since when the city archive has been the sole user of the castle.