It is situated at Jalan Tukang Emas (Goldsmith Street), also known as "Harmony Street" because of its proximity to the Sri Poyatha Moorthi Temple and Cheng Hoon Teng Temple.

The original structure built by Indian Muslim traders in 1748 was a wooden building and in 1872, it was rebuilt with bricks.

The architectural design of the mosque is a cross between Sumatran, Chinese, Hindu, and the local Malays.

The minaret (resembles a pagoda), ablution pool and entrance arch were built at the same time with the main building.

The mosque also has a blend of English and Portuguese glazed tiles, Ionic columns with symmetrical arches in the main prayer hall, a Victorian chandelier, a wooden pulpit with Hindu and Chinese-style carvings, and Moorish cast iron lamp-posts in the place of ablution for pre-prayer cleansing.