[6] Despite his dislike for Ishida Mitsunari, Mori Takamasa initially remained loyal to the Western Army and led his forces at the Siege of Tanabe in 1600.

He defected to the Eastern Army shortly after the Battle of Sekigahara at the persuasion of Tōdō Takatora,and through his intercession was relocated by Tokugawa Ieyasu to the 20,000 koku holding of Saiki Domain in Bungo Province in 1601.

The Kusu area was given to the Kurushima clan, who were of a branch of the Murakami pirates who dominated the Seto Inland Sea during the Sengoku period.

As their assigned kokudaka was only 14,000 koku, they were not permitted the prestige of having a castle, but were forced to abandon it and construct a jin'ya fortified residence at the base of the mountain as the stronghold of Mori Domain.

As entry to the castle site was banned until the Meiji restoration, the remnants of stone walls, dry moats and some foundations for yagura turrets are relatively well preserved.