The ground of the Orangerie was an old garden, in which Landgrave William IV built a small summer house.
The baroque castle of today was designed and built by the architect Johann Conrad Giesler in 1702, who worked on the court of the landgrave.
In World War II, the Orangerie was damaged and later repaired to host the federal garden show 1955.
After the show the building was restored and transformed into the astronomy and physical cabinet with planetarium; in 1996 an astronomical garden was added.
During the documenta 12, the space in front of the Orangerie was used for the exhibition, which included a temporary 9,500 square meter hall named Auepavillon.