During the colonial era the palace became favorite residence of the governors-general due to Bogor's more adaptable climate.

[1] The original colonial building on the site of Istana Bogor was a mansion named Buitenzorg, meaning "without a care" in Dutch (also Sans Souci, meaning "without a care" in French), which dated back to 1745 as a country retreat for the Dutch governors to escape the heat and diseases of Batavia.

The location for the new palace was discovered by Baron van Imhoff on 10 August 1744, in a village named Kampong Baroe.

On the site he ordered a mansion to be built, however the construction was not completed by the end of his term in 1750, and thus it was continued by his successor Jacob Mossel.

[3][4] Later the new governor general Baron van der Capellen (1817–1826) added a small dome on the roof of the main building and founded the botanical garden next to the palace ground.

[5] The ruined palace was then demolished and rebuilt into its present form in 1856— this time with only one storey instead of the original two, as a precaution against further earthquakes.

[3] From 1870 to 1942 the Istana Bogor served as the official residence of the Dutch governor general, however state affairs were still largely conducted in Batavia.

[3] The palace is decorated with an extensive art collection (448 paintings, 216 sculptures and 196 ceramics), 90% of which were accumulated by Sukarno.

Paviliun Dyah Bayurini was built in 1964 and is used by the head of state and his or her family during a stay in Bogor, the pavilion also include a swimming pool.