Almost immediately after the Battle of Sekigahara, Tokugawa Ieyasu order the destruction of Ōtsu Castle and the construction of a new castle at this location in order to control the Tōkaidō highway connecting Kyoto with Edo and the provinces of eastern Japan.

On its completion, the castle was assigned to Toda Kazuaki, who was appointed daimyō of the newly created Zeze Domain with a kokudaka of 30,000 koku.

The castle was severely damaged by an earthquake in 1662, and due to its lakeside location suffered from land erosion which required constant upkeep.

The site of the inner bailey has become the "Zeze Castle Ruins Park", with a few stone walls remaining a reconstructed gate.

One of the two-story corner yagura towers survives at Chausuyama Park in Ōtsu, where it has been converted into a building for use for assemblies, but it no longer retains its original form.