[2] The five immortals honored by the temple are first attested in a passage of the 983 Taiping Imperial Readings (太平御覽) explaining Guangzhou's nickname "City of the Rams".

[3] One of the etiological myths given is that At one time, Sun Hao made Teng Xiu a governor but, before reaching his prefecture, there were five immortals riding five colored sheep and bearing the five grains who came upon him and, after he received them, they left.

[3] Later accounts of the immortals' local cult made them culture heroes who introduced rice farming to Guangzhou around the time of its founding,[5][6] which occurred historically by the Qin army under Zhao Tuo in 214 BC.

An abbey and shrine to the five immortals is first attested during the early Song (c. 11th century), with rulers making sacrifices and locals offering prayer to them.

It was purchased by a local club, who deposed the residing Taoist clergymen, but its site was given special protection by the municipal government and was eventually converted into a school.