Situated on the shore of Zürichsee lake, Einsiedlerhaus is the eastern extension of the fortification of a neighbouring Capuchin friary (German: Kapuzinerkloster) in the so-called Endingen area, located below the Lindenhof hill.

Historians mention a 10th-century ferry station located there – in 981 AD as well as the vineyard on the Lindenhof hill – between Kempraten on Kempratnerbucht, Lützelau and Ufenau islands and presumably present Hurden, which allowed the pilgrims towards Einsiedeln to cross the lake before the prehistoric lake crossing at the Seedamm isthmus was re-built between 1358 and 1360.

Nevertheless, the fortress section with the adjoining semi-circular tower Endingerturm, since 1597 with the passage in the present-day Capuchin monastery garden, is the most historically significant remain of the former city fortifications.

As the Capuchin friar Christian Endres, the last woolen cloth weaver in the Rapperswil monastery, died in 1971, the monastic tradition extinct after exactly 300 years.

[2] In compensation, an agreement dated 6 February 1975 between the Einsiedeln Abbey (owner), the city of Rapperswil (leaser) and the Capuchin monastery (former leaser), certifies that Rapperswil solemnly replaces the Capuchin's fruit loss in natura – every autumn a fruit farmer from neighbouring the Feldbach village delivers 50 kilograms (110 lb) apples, with the consent of the Benedictine monastery Einsiedeln as owner of the building, for which the Capuchins have to deliver a symbolic annual loan of one Swiss Franc.

After a fire, the building got its present appearance in 1717,[1] as well as probably the adjoint garden's walls which separate it from the Endingerstrasse lane respectively lake shore, and the neighbouring vineyard Schlossberg.

Until 1610 the present gate tower and Einsiedlerhaus formed a bulwark that was protecting the inner harbor, now Fischmarktplatz, flanked by palisades at the westerly Endingerhorn where in the 1660s a lakeside battery was built.

Haus der Musik is a youth music school provided by the proMusicante association and started teaching in September 2012.