After Toyotomi Hideyoshi’s successful conquest of Kyūshū, local warlord Matsura Shigenobu was granted Hirado County and the Iki Island to be his domain.

However, he burned the castle down himself in 1613, as a gesture of loyalty towards Shōgun Tokugawa Ieyasu, having served in the losing Toyotomi side during the Battle of Sekigahara.

It was intended to be the keystone in coastal defense in the East China Sea region, as the government had by then implemented a policy of national seclusion against Western traders and missionaries.

In 1871, with the abolition of the han system, all structures of Hirado Castle were dismantled, with the exception of the northern gate, a yagura and the moat, and the grounds turned into Kameoka Park, with a Shinto shrine dedicated to the spirits of the successive generations of the Matsura hankang.

One of these artifacts is a 93-cm long Japanese sword (tachi) dating from the Asuka period, and is locally purported to have been carried by a general during the time of the legendary Empress Jingū's invasion of Korea.