During the late Muromachi and Sengoku period, this area came under the control of the Imagawa clan, a powerful warlord from Suruga Province.

However, the greatly weakened Imagawa clan was unable to withstand the combined forces for Tokugawa Ieyasu from Mikawa and Takeda Shingen from Kai.

Under Horio Yoshiharu, the castle was renovated in line with contemporary advances in stone ramparts, and was expanded in size.

After the Battle of Sekigahara, the Horio clan was relocated to Izumo Province and Hamamatsu was briefly ruled by Tokugawa Yorinobu, followed by a succession of fudai daimyō through the remainder of the Edo period.

Many also went on to hold high offices within the shogun administration, including five rōjū, two Kyoto Shoshidai, two Osaka-jō dai and four Jisha-bugyō.

The central portion was retained by the city of Hamamatsu as a park after World War II In 1958, a faux donjon was constructed out of reinforced concrete on top of the original stone palisade built by Tokugawa Ieyasu.

There is a small museum inside which houses armor and other relics of Tokugawa clan, as well as a miniature model of how the city might have looked at the start of the Edo period.