[2] By the 1380s they had acquired land in Domleschg and around Heinzenberg Castle, which brought the counts into conflict with the Bishop of Chur.

The decades long conflict between the Bund and the bishop weakened the power of the Rhäzüns family and forced them to join the expanded Grey League in 1424.

Five years later, in 1473, the inheritance was finally settled and the castle as well as the extensive Rhäzüns lands passed to Niclas von Zollern.

In 1497 the pro-French Count Trivulzio tried to buy the castle, but was countered by the Habsburgs who acquired it along with membership in the Grey League.

Two years later, during the Swabian War, Three Leagues troops occupied the castle to prevent it from becoming a Habsburg stronghold.

However, the Congress of Vienna gave the castle to the Canton of Graubünden in the following year, and it finally was fully handed over on 19 January 1819.

[2] The inner gate tower and the northern residential wing were built in the 14th century and still show frescoes of the coat of arms of the Barons as well as a hunting scene.

During the 16th century the east wall was added after the destruction of the eastern tower and the rest of the castle was repaired and renovated.