The oldest part of the castle is built on the isolated rock on the last hill of the Low Tatras jag with an altitude of 375m above sea level.

The cause was settled in the castle on 3 August 1263 with attendance of highly placed city dignitaries and clergy, to whom belonged also the Ľupča monastery principal, guardian Thomas.

After his death his successor Stephen V visited the castle only twice, as it is reported in documents, in August 1270 and one year later when he confirmed the privilege to comes Ondrej (vogt of Banská Bystrica) for possession of part of the land.

When Ladislaus IV was recognised as an adult, he visited the castle in 1278 and granted Poniky land to master Filip for his services in the battle on the Marchfeld against the Czech monarch Ottokar II of Bohemia.

In that period the castle history is connected with the name of the last district mayor of the Zólyom County, magister knight Donč.

In 1380 Louis I granted Brezno's guests the "Siavnica right", marked the chotar and allowed the free vote for mayor and priest.

After one year the Brezno's representatives visited king Louis I in Ľupča castle again to ask for confirmation of their rights by the royal charter.