The museum itself is housed within a distinctive green-hued mansion at Church Street, George Town, which once served as the residence and office of a 19th-century Chinese tycoon, Chung Keng Quee.

The site was formerly used by the Ghee Hin, a Chinese secret society that was a rival of Chung's Hai San.

The Straits Eclectic-style mansion, originally painted white, incorporated both European and Chinese designs.

A five foot way was indented along the building's side facade, while the interior contains a number of wide courtyards similar to the Chinese townhouses of the era.

Chung also acquired European interior furnishings for his new mansion, such as Scottish cast iron works and Stoke-on-Trent ceramic floor tiles.