It is known for its Japanese garden, which was designated a NationalPlace of Scenic Beauty in 1974, with the area under protection expanded in 2009 to include the Main Hall and Shoin of the temple.

It was revived in 1339 when Ashikaga Takauji designated it as the official Ankoku-ji temple dedicated to the memory of the dead of the Genkō War of 1331-3 for Bitchū Province.

During the Eishō era (1504-1521), the lord of Bitchū Matsuyama Castle, UenoYorihisa, took the tonsure at this temple, and after his death it was renamed for him, using the kanbun reading of the kanji "頼久", to become Raikyū-ji.

After the Battle of Sekigahara in 1600, Tokugawa Ieyasu assigned Kobori Masatsugu as daikan to administer confiscated easternmost part of the Mōri territory in Bitchū.

At this time, Bitchū Matsuyama Castle was in ruins, so Masakazu conducted government affairs at Raikyū-ji until the caste reconstruction was completed in 1619.