Today, only a few parts of Okayama Castle's roof (including the fish-shaped-gargoyles) are gilded, but prior to the Battle of Sekigahara the main keep also featured gilded roof tiles, earning it the nickname Golden Crow Castle (金烏城, Kin U-jō).

Three years later, Hideie sided with the ill-fated Toyotomi Clan at the Battle of Sekigahara, was captured by the Tokugawa Clan and exiled to the island prison of Hachijo.

The castle and surrounding fiefdoms were given to Kobayakawa Hideaki as spoils of war.

Kobayakawa died just two years later without leaving an heir, and the castle (and fiefdom) was given to the Ikeda Clan, who later added Kōraku-en as a private garden.

The reconstructed main keep is a concrete building complete with air-conditioning, elevators and numerous displays documenting the castle's history (with a heavy focus on the Ikeda era.)