The castle was first mentioned in writing in 1294 as Palocha and was built by a certain Ditrich (or his son Arnold) from Spiš to secure a ford on the road to Poland.

The border castle became royal property and was given to the Drugeth family by Charles I in 1317.

Between 1449 and 1458, the castle was ruled by the Hussite Bohemian brothers and their leader Petr Aksamit.

This family had the castle rebuilt in the 16th century in the Renaissance style, followed by another extensive reconstruction in the 17th century, the result of which was a residential castle that replaced the medieval fortress.

The entire complex fell victim to a fire in 1856 and the Palocsay family died out a year later.