It initially rented out the first floor of the Ballarat Academy of Music; the current building on Lydiard Street North opened in 1890.

The gallery was privately owned until financial insecurity led to the building and collection being handed over to the Ballarat City Council in 1977.

In June 1884, the prominent Ballarat benefactor James Oddie (1824–1911) funded and lent paintings for a public exhibition at the city hall.

Lacking their own building, they rented out the first floor of the Ballarat Academy of Music—where Her Majesty's Theatre now is—from Sir William Clarke.

[3] It was designed by Tappin, Gilbert and Dennehy in the Renaissance Revival architecture style as a bluestone brick and render facade and stone stairway.

The foundation stone was laid by Sir William Clarke on 21 June 1887 to commemorate the Golden Jubilee of Queen Victoria.

In 1978, the gallery was in a large amount of debt;[8] it had begun transferring the building and the collection to the city council the year before.

[10] According to a member of the Gallery Association Council at the time, Radford received a ransom message several months later and the painting was retrieved from a park in Sydney after it was paid.

[12] Listed on the Victorian Heritage Register, the gallery is a double-storey building constructed in the Renaissance Revival architectural style.

Inside, a large stone stairway with Art Nouveau ceiling decoration leads up to the main gallery rooms.

[4] Designed by the architectural firm Tappin, Gilbert and Dennehy and constructed between 1887–1890,[4] it is the oldest purpose-built art gallery building in Australia.

[10] Items such as Old Ballarat as it was in the summer of 1853-54 (1884), for which the gallery commissioned artist Eugene von Guerard, depict the city's development.

Unusually for a regional gallery, it has been acquiring Indigenous Australian art since the 1930s, particularly from the Northern Territory's Top End, which it includes as part of its contemporary collection.