During the Muromachi period, the area was dominated by the local Miyoshi clan, who also controlled Kawachi Province across the Kii Channel.

In 1582, the warlord Chōsokabe Motochika of Tosa Province attempted to united the island of Shikoku under his rule, and defeated the Miyoshi.

For these reasons, at the time of the Battle of Sekigahara, Ishida Mitsunari forced Hachisuka Iemasa to take the tonsure and forcibly exiled him to Mount Koya and the castle was occupied by troops from the Mōri clan.

The Meiji government ordered the destruction of most of the castle structures in 1873, and by 1875 only one gate survived.

To commemorate the victory of Japan in the Russo-Japanese War in 1905 the remaining castle grounds were opened to the public as a park.

The remaining one original gate of the old castle was destroyed during World War II on July 4, 1945.