Miki Yoritsuna (1540-1587) who ruled the southern half of Hida gradually expanded his domains with the support of Oda Nobunaga.

After Nobunaga's assassination in 1582, he decisively defeated an alliance of the remaining local lords united Hida province, with his base at Matsukura Castle.

However, Yoritsuna supported Sassa Narimasa in neighboring Etchū Province against Toyotomi Hideyoshi and was destroyed by the forces of Kanamori Nagachika (who was then the ruler of Echizen Ōno Castle).

Hideyoshi awarded Hida Province to Kanamori, who started to build a new castle on Shiroyama mountain in Takayama in 1588.

Three years later Hida Province became tenryō territory, under the direct control of the Tokugawa shogunate, and the buildings of the castle were pulled down, with a jin'ya built at the foot of Shiroyama.