The original mosque was constructed in 630 Common Era, at the site where a demolished temple, the Kaaba-type shrine of the goddess al-Lat once stood.

[4][5] A cemetery for the martyrs of those in the siege of Banu Thaqif was established in the same year, on the eastern side of the mosque.

[4][5] In 1813, the Swiss traveller Johann Ludwig Burckhardt, who was visiting the Arabian Peninsula just after the takeover of the city by Muhammad Ali Pasha, described the mosque as being in ruin, while the domed mausoleum over Ibn Abbas' tomb had been destroyed by the Wahhabis.

[4][5] After Ibn Saud had taken control of the whole of Saudi Arabia, the mausoleums that still existed in his territory, including Mecca and Medina, were demolished.

The mosque and mausoleum were demolished and then rebuilt in 1958, this time the grave was returned to its original state without a dome over it.