The shrine originally enshrined Tokugawa Ieyasu in the center, flanked by the Sannō Gongen on the left and Matajin on the right (two protective spirits), and was under the control of a Buddhist temple, Daishō-ji.

A chapel to Tokugawa Hidetada was added in 1624 and since then the deified spirits of the successive shoguns have been enshrined at the shrine.

The name of the shrine's location was changed to "Tokiwayama" by Tokugawa Mitsukuni in 1699.

This action was cited as one of the reasons for his dismissal from his posts within the shogunate administration and house arrest at Mito.

The Honden of the shrine, which had been designated as an Important Cultural Property since 1917 was destroyed by American bombing in World War II.