The Roosenburgs purchased the castle in 1957, after which time they made extensive renovations, replacing the 1848-built chalet roof with a flatter design.

He rented the castle, furnished with nothing but straw beds and a fruit crate, for fifty guilders per month.

[12][13] Early members of the community included Roosenburg, his painter girlfriend (and later wife) Jopie Goudriaan, and the sculptors Hanni Rädecker and Piet Damsté.

Nicolaas Wijnberg [nl], writing in 1997, recalled that most of the building's twenty-one rooms remained empty when he arrived at the castle in 1942 with another painter.

[16] Members of the colony couriered funds for the Resistance and used the castle's proximity to Belgium to facilitate the emigration of Jewish refugees.

As narrated by Wijnberg, they "put on strange hats and caps, painted [their] shoes, and when [they] went outside [they] deliberately stared a lot and dreamily at the landscape and the cloudy sky".

By 1942, Jopie Roosenburg had painted a mural – a reproduction of a Picasso composition with mandolin, compotier, and melon – in a hallway.

[14] Much restoration work had been completed by Teun Roosenburg, Rädecker, and Damsté by war's end in 1945, after which the latter two returned to their native Amsterdam.

[16] In a 1991 interview, Hans van Norden [nl] described the ambience of the castle under the Roosenburgs as "special ... you literally tripped over the easels.

[17] Expansions are being constructed to restore the original design and expand the complex to eighty rooms, with an expected ceremonial opening date of 28 August 2024.