According to legend, the Great Southern Mosque was founded during the Tang dynasty, but there are no known records to confirm this.

The mosque was damaged severely during the Cultural Revolution with many of the historical artifacts it housed being destroyed and the building converted into a factory.

[2] The basic layout of the mosque follows the courtyard style of a traditional Chinese temple with major buildings arranged along a main axis of symmetry.

In keeping with Chinese tradition, the main entrance of the mosque is protected by a spirit wall.

[2] The mosque is located in Lixia District of Jinan, to the west of the historical city center and the Baotu Spring Park.