The first settlement of the castle cliff is dated to Púchov culture, also a Celtic coin was found on the hill.

For the shift from wooden castle is probably responsible Béla IV, after the invasions of Batu Khan.

During last few years of the 14th century, King Sigismund gave the fortress to palatine Sudivoj, but the owner was quickly changed, and from 1400 to 1425 it belonged to Stibor of Stiboricz.

His diligent work for the ruler prepared a good position for his sons and great upsurge of his possession.

He was one of the leaders of the Counter-Reformation in the times when Martin Luther's influence just started to reach the country.

After the great burn out of his "eagle's nest" Rafael did not feel safe, so he made an agreement with Habsburgs.

His loyalty was approved when he was named governor of the northern part of Trencsén County region.

The marble monument of Rafael Podmanitzky in Považská Bystrica is of interest because it has an inscription all around in old Slovak language it says "LETA PANIE TISICEHO PIETISTEHO PADESATEHO OSMEHO PRWNI STRZEDU W PUSTIE UMRZEL GEST UROZENY PAN RAFFAEL PODMANITZKY Z PODMANINA JEHOZ DUSSY PAN BUOH MILOSTIW RACZ BIETI".

The castle was shelter for the Balassa family during an uprising of Francis II Rákóczi against the Habsburgs.

From this moment the castle was left of peoples interest until it was bought in 2007 by Považská Bystrica and voluntary organization "Zdruzenie na zachranu Povazskeho hradu " did not start essential reparations on it.

The castle with shingle roof belonged in times of his fame to one of biggest well-fortified buildings of that kind.

Here was probably also a secret gate through which people from the castle were supplied in cases of attack, when the main entrance was built-in.

An organisation called " Zdruzenie na zachranu Povazskeho hradu " was established in 2008 to protect the county's heritage from being destroyed.

The castle was, with its famous rulers Ladislav & Rafael Podmanitzky, often the theme of legends and narratives.

In spite of the great effort of people for whom the destiny of the country's heritage is not vacant, there are still several cases of vandalism on the castle.

Building works on the manor started in the first third of the 17th century and were finished 1631, when family moved from Castle.

Last owner was Baroness Popper von Podhragy, who supported reconstruction of the manor until after the World War I in 1919 it became possession of the state.

In the vicinity of the manors was a cemetery with gravestones of Balassa family and garden with plants brought from around the world.

In the premises of the church is statue of Saint John of Nepomuk, which was coincidentally brought to Považské Podhradie on abandoned flatboat during floods in 1784.

It was rebuilt in 1940 to satisfy the need for capacity, leaving only presbytery, tower and the northern part of the walls of the old church.

Nowadays paned windows were designed by Slovak artists Vincent and Viera Hloznik.

In the entrance hall of the church is the gravestone of Rafael Podmaitzky, Zsigmond Balassa and his wife Alżbeta Zborowska.

On the opposite bank of the Váh River, there lies narrowest canyon in Slovakia called Manínska tiesňava.