[3] Following several conflicts with the neighbouring East Slavic states of Novgorod and Pskov, the Danish rulers started building a stone stronghold at the beginning of the 14th century as a border defense.

At the beginning of the 14th century, a small forecourt was established at the north side of the stronghold and, in the middle of the century, a large forecourt was added to the west side, where citizens were allowed to hide in case of war, as the town of Narva was not surrounded by a wall during Danish rule.

The stronghold has for the most part preserved the original ground plan, with its massive wings and a courtyard in the middle.

The Hermann Tower was also completed by the Livonian Order, necessitated by the establishment of the Ivangorod fortress by Muscovite Russia on the opposite side of the Narva river in 1492.

Towards the end of the rule of the Livonian Order in the 16th century, the town wall was supplemented, the gates were strengthened by adding foregates, and several wall towers were adapted to use as special cannon towers or rondels, two of which can be seen today in their reconstructed shape in the corners of the castle's western court.