The building was a private mansion built in 1819[1] by Aaron Cardozo, a prosperous merchant of Jewish Portuguese descent who had settled in Gibraltar, as his family home.

However, as he had been a close friend of Lord Nelson and had supplied his fleet, he was eventually granted a site to build a house in the Alameda on the condition that it be "an ornament" to the square.

[1] In 1922, his son Pablo Larios, Marquis of Marzales (Master of the Royal Calpe Hunt for 45 years), sold the building to the Gibraltar colonial authorities, which intended to turn it into a post office.

In 2015, HM Government of Gibraltar set up the Mario Finlayson National Art Gallery at the City Hall.

The gallery exhibits the works of prominent Gibraltarian artists such as Gustavo Bacarisas, Jacobo Azagury, Leni Mifsud, and Rudesindo Mannia.