Throughout the middle to later Edo period, Komine Castle was home to the Abe clan, daimyō of Shirakawa Domain.

South of the inner bailey, protected by a wide water moat, was a secondary enclosure approximately 200 x 100 meters which contained the residence of the lord.

During the Boshin War, Shirakawa was a stronghold of the pro-Tokugawa Ōuetsu Reppan Dōmei and came under attack during the Battle of Aizu by the pro-imperial army of the Satchō Alliance.

Although the invading army was outnumbered, it had superior artillery, and Komine Castle had been designed primarily to defend against attacks from the north, so its southern approached were more lightly fortified.

Many of the structures of the castle were destroyed during the battle, and much of what remained, including a large section of its stone walls, were later pulled down following the abolition of the han system.

An earth wall with uneven stones made up the original base of Komine Castle before it collapsed in the 1970s due to rain.