Built in the 14th century, it served as the seat for administrators of property of the Warmian Cathedral Chapter.

[2] The most well-known administrator caretaker was Nicolaus Copernicus, a canon of Warmian Cathedral Chapter in Frombork, who resided here between 1516 and 1521.

[4] It is made up of one wing to the north-east of the courtyard and was initially surrounded by a line of fortifications and a ditch which lead to the Łyna River with a bascule bridge.

During the Thirteen Years' War (1454–66), the castle was being captured multiple times by both powers, and afterwards became part of the autonomous province of Royal Prussia within the Crown of the Kingdom of Poland.

[7] Over time, the castle's militaristic importance diminished and its role for the housing of the Bishops of Warmia became less favourable.

Between 1901 and 1911, the castle underwent various renovations; this included the change in floor levels, and arched windows in the cloister.