The schloss lies northwest of the town centre of Freyung on a rocky crag surrounded on three sides by the Saußbach river.

Wolfstein Castle was built as a fortification by the Bishop of Passau, Wolfger von Erla, around 1200.

In 1301, a place is first mentioned in a document as Purchstol zu Wolferstein und ein Wald dazu, later Freyung.

Since the incorporation of the municipality of Ort on 1 April 1954, Wolfstein Castle has stood on the territory of the town of Freyung.

The schloss ensemble includes a mighty tower house, built between 1199 and 1204 and 1590, two low wings with arcades, dating to the late 16th and early 17th century, and the castle wall with closed quadrangular courtyard.