[1] The castle was built on a rocky outcropping above the village probably in the 12th century as the home of the Greifenstein family.

In 1243, his relatives Heinrich and Albert were listed as witnesses in a legal proceeding as Hainricus et Albertus de Grifinstain.

[5] By 1392 the von Matsch family were robber knights who used the castle as a base for raids against the Bishop's estates.

In 1394 Bishop Hartmann II successfully sent an army to drive out the robbers and take control of the castle.

In 1421 Duke Ernst of Austria negotiated a compromise where by the Bishop kept both Greifenstein and Tschanüff Castles but had to pay the Matsch family 2,500 Marks.

[2] When the roof finally collapsed, part of the castle was broken up for stone for the new school house in Filisur.

Much of the massive curtain wall that surrounded the middle castle is still standing on the north and western sides of the terrace.