In 1046, Henry III of the Holy Roman Empire gave the burghers of Colditz permission to build the first documented settlement at the site.

Forests, empty meadows, and farmland were settled next to the pre-existing Slavic villages Zschetzsch, Zschadraß, Zollwitz, Terpitzsch and Koltzschen.

During the Middle Ages, the castle was used as a lookout post for the German Emperors and was the hub of the Reich territories of the Pleissenland (anti-Meißen Pleiße-lands).

During this period the portal at what is known as the church house was created during 1584, made of Rochlitz Porphyr (rhyolite tuff) and richly decorated in the mannerist style by Andreas Walther II.

It was at this time that both the interior and the exterior of "the Holy Trinity" castle chapel that links the cellar and electors' house with one another were redesigned.

The castle was used by Frederick Augustus III, Elector of Saxony as a workhouse to feed the poor, the ill, and persons who had been arrested.

When the Nazis gained power during 1933, they converted the castle into a political prison for communists, homosexuals, Jews and other people they considered undesirable.

[2][3] Since the castle is situated on a rocky outcrop above the River Zwickauer Mulde, the Germans believed it to be an ideal site for a high security prison.

[4] While the camp was home to prisoners of war from many different countries, including Poland, France, Belgium, the Netherlands, and Canada, in May 1943 Wehrmacht High Command decided to house only British and American officers.

One escape scheme even included a glider, the Colditz Cock, that was built and kept in a remote portion of the castle's attic during the winter of 1944–45.

Assigned to the task force that liberated the castle, American war correspondent Lee Carson entered Colditz on 15 April 1945 and took the only photograph of the glider completed in the attic.

During 1999, a full-sized replica of the glider was commissioned by Channel 4 Television in the UK and built by Southdown Aviation Ltd at Lasham Airfield, closely following Goldfinch's drawings.

With renovations largely completed, the castle now includes both a museum and guided tours showing some of the escape tunnels built by prisoners of the Oflag during the war.

The chapel has been restored to its prewar decoration, with glass panels inserted to the flag stone flooring to reveal an escape tunnel dug by French escapees.

The outer courtyard and former German Kommandantur (guard quarters) have been converted into a youth hostel / hotel and the Gesellschaft Schloss Colditz e.V.