It was built on a slope of Vistula River by Casimir III the Great and extended in the 16th century.

The remnants of the Gothic structure are visible in the foundations of the octagonal tower of the south corner, which is the oldest part of the monument.

The existing tower was built during the reign of Casimir IV Jagiellon in the 15th century as an integral part of the so-called Great House, the seat of the prince.

The construction was supervised by the Royal architect Benedyct Sandomierski, who erected two-storey arcaded cloisters around a closed courtyard.

The survived western wing of the castle was later rebuilt during the reign of king John III Sobieski between 1680-1688.