Archaeological excavations have found evidence that the island was a ritual location in the Yayoi period, and unverifiable legends state that the worship of Ōyamatsumi came from Baekje on the Korean Peninsula and came to Japan from the Japanese domination of Mimana under Emperor Nintoku.

Due to this practice, the shrine has an enormous collection of weaponry, estimated by some to include approximately 40% of Japan's arms and armor designated as either a National Treasure or an Important Cultural Property, including eight National Treasures and 76 Important Cultural Properties (as of 2014).

The priesthood of the Ōyamazumi Shrine was hereditary to the Ochi clan, who had been the Kuni no miyatsuko of Iyo since the Kofun period.

The GHQ authorities were disturbed by the vast number of swords, and ordered the disposal of all items except those which had already been designated as National Treasures.

The shrine authorities responded by secretly burying them and later, after the end of the occupation and the establishment of the Japan Self-Defense Forces, they returned the swords to the Japanese government.