Supposedly, the castle was built by the brothers Adolf and Everhard von Berg around the year 1108 after Henry V granted them land in Sauerland for their loyal services.
They continued to only occasionally inhabit Altena Castle and from 1392 onward it was only used as a residence for the county bailiff (Amtmann).
Count Engelbert III of the Mark gave the small settlement at the base of the mountain the rights of liberty (such as self-governance).
There was a controversial debate about the modes of reconstruction, where the preference of historic designs over the medieval and early modern architecture was criticised.
In 1914, Richard Schirrmann established the world's first youth hostel within the castle, which is still in use today (Jugendherberge Burg Altena).