The area then came under the control of Oda Nobunaga, who assigned it to one of his generals, Kawajiri Hidetaka.

After Nobunaga was assassinated in the Honnō-ji incident, the territory came under the control of Toyotomi Hideyoshi, who assigned Hineno Takayoshi as daimyō of Suwa Domain.

The Hineno were reassigned in 1601, and the domain was returned to the hands of the Suwa clan by Tokugawa Ieyasu.

Following the establishment of the Meiji government and the abolition of the han system, the remaining structures of the castle were dismantled in 1875, leaving only the stone foundations.

A Shinto Shrine dedicated to the war dead was established within the grounds in 1900 and the area of the Second and Third Baileys was built over as a residential district.