At the end of the Edo period, Tatebayashi Castle was home to the Akimoto clan, daimyō of Tatebayashi Domain, but the castle was ruled by a large number of different clans over its history.

According to legend, Akai Terumitsu saved a young fox from naughty children, and then in the evening an Inari appeared and recommended a location for his castle, drawing a design for the fortifications on the ground by its tail.

After Tokugawa Ieyasu took control over the Kantō region in 1590, he assigned Sakakibara Yasumasa, one of his most trusted Four Generals as daimyō of Tatebayashi, with revenues of 100,000 koku.

The area was strategically important for its control over the Tone River which supplied Edo, and after the Sakakibara clan was reassigned to other territories, the castle was retained by the most trusted of Tokugawa retainers or relatives, including at one point the younger brother of Shōgun Tokugawa Ietsuna, the future shōgun Tokugawa Tsunayoshi.

However, from the death of Tsunayoshi’s son, Tokumatsu, in 1683, the donjon of the castle was allowed to fall to ruin.