The schloss is situated on the banks of the Spree river, near the Berlin Victory Column, along the northern edge of the Großer Tiergarten park.

There were preexisting structures on the site, including the manor house which King Frederick's architect Georg Wenzeslaus von Knobelsdorff had built for himself in 1743, which was demolished, and a leather factory on the Spree river waterfront which was converted into the right side-wing.

The palace was named Bellevue as its view reached the tower of Schloss Charlottenburg before the viaduct of the Berlin Stadtbahn was built nearby in the 1880s.

The only room that kept its original decoration is a ballroom on the upper floor designed by Carl Gotthard Langhans, the architect of the Brandenburg Gate.

The Free State of Prussia acquired the property from the former Emperor in 1928 and used it as a museum of ethnography during the 1930s before being renovated as a guest house for the Nazi government which had purchased it in 1938.

In 1945, according to testimony reported in the 1995 documentary film On the Desperate Edge of Now, Berlin citizens buried statues of historical military figures from the Großer Tiergarten in the grounds of the palace to prevent their destruction.