During the Battle of Odawara in 1590, Toyotomi Hideyoshi dispatched an army led by Uesugi Kagekatsu and Maeda Toshiie and destroyed Wada Castle.

However, in 1597, Ieyasu ordered Ii Naomasa to construct a new on the site of the ruins of Wada Castle, as the location controlled a strategic junction connecting the Nakasendō with the Mikuni Kaidō highways.

In 1619, Andō Shigenobu began an ambitious reconstruction project, which lasted for 77 years over the following three generations, which included a three-story donjon in the center and two-story yagura at each of the cardinal directions.

Prior the end of World War II, most of the former castle grounds were occupied by the Takasaki-based Imperial Japanese Army's 15th Infantry Regiment.

The shōin where Tokugawa Tadanaga committed suicide is now located on the grounds of a nearby Buddhist temple, Chōshō-ji, where it serves as the priest's residence.