However, after the 1590 Siege of Odawara, Toyotomi Hideyoshi forced Ieyasu to exchange his holdings for new territories in the Kantō region and gave Kai Province to his Kato Mitsuyasu (1537–1593).

Kōfu Castle was originally T-shaped, with the main body spreading on the ridge of the hill from east-to-west, separated into three or four layers of terraces over a 200 by 100 meters area.

The inner bailey was at the top of the hill measuring 100 by 50 meters, with a four-story tenshu painted black with gold-colored roof tiles.

Although the tenshu constructed by Asano Nagamasa had fallen into ruin and was not replaced, the castle itself was kept in good repair and was ruled by a cadet branch of the clan.

Yanagisawa Yoshiyasu was also a descendant of the Takeda clan, and made strong efforts for restoring the prosperity of the region.

However, his son Yanagisawa Yoshisato was transferred to Yamato Kōriyama Domain, and Kai province became tenryō territory ruled directly by the shogunate though an appointed hatamoto administrator.

[2] During the Boshin War of the Meiji restoration, Katsu Kaishū dispatched the Shinsengumi under Kondo Isami to seize Kōfu Castle before it could fall into the hands of the advancing Imperial army.