The castle was first constructed at the end of the Heian period when the descendants of Taira Kanesada, who was appointed as the benzai envoy (supervisor of the manor) of Shimazu shōen, came to this area and settled.

Kanesada's son, Kanetoshi, took the name of "Kimotsuki", and his descendants ruled for 18 generations, or over 400 years until the end of the Sengoku period.

The Kimotsuki clan continued to rule, albeit as retainers of the Shimazu, over significantly reduced territory until 1580, when they were transferred to Ata (present-day Kinpō-chō, Minamisatsuma), and Kōyama Castle was abandoned.

[2] Traces of the Honmaru Inner Bailey, which was located at an elevation of 82 meters on the mountain, as well as secondary enclosures divided by dry moats and earthworks remain and can via viewed by a hiking trail.

The site is approximately one hour by car from Shibushi Station on the JR Kyushu Nichinan Line.