Situated at the highest point in Bangalore, High Grounds (3031 feet above sea level), it was built by Sir Mark Cubbon between 1840 and 1842 when he was the Commissioner of Mysore territories of the British.

A ballroom was constructed in 1874 when the King Edward VII, Prince of Wales, visited India.

In 1881, when power over the territory was transferred back to the Mysore royal family, the office of the Commissioner was abolished.

The Constitution of Independent India created the office of the "Raj Pramukh" (Governor).

The garden is fringed with century old pine and fir trees, and has an artificial waterfall.