The island is formed from a cluster of eighteen lava domes, with rhyolite and pyroclastic ash deposits.

The highest of these lava domes, Tenjō-san (天上山), has a height of 571 metres (1,873 ft), and was last active in 838 AD per the ancient Japanese history Shoku Nihon Kōki.

The island has been recognised as an Important Bird Area (IBA) by BirdLife International because it supports populations of Japanese wood pigeons, Tristram's storm petrels, Japanese murrelets, Ijima's leaf-warblers, Pleske's grasshopper warblers and Izu thrushes.

[4] Kōzu-shima has been inhabited since at least the Japanese Paleolithic era, and archaeologists have found Jōmon period stone tools made from obsidian in Shizuoka and Yamanashi Prefectures.

Under the Ritsuryō system of the early Nara period, the island was part of Suruga Province.