[2] His descendants formed cadet branches named after their residence, and through the Muromachi period, Iwamura was ruled by the Tōyama clan.

However, Oda Nobunaga was eager to build a relationship with the Tōyama, primarily as they bordered the lands of his arch-nemesis, Saitō Dōsan of Gifu.

Per this treaty, her adopted son (Oda Katsunaga) was sent to Kai Province as a hostage, and she was compelled to marry Akiyama Nobutomo.

The remaining Tōyama forces joined the Takeda, a move which enraged Oda Nobunaga, who attacked Iwamura Castle on repeated occasions.

In 1575, following the decisive defeat of Takeda Katsuyori at the Battle of Nagashino, Oda forces surrounded Iwamura Castle, but it continued to hold out.

In 1582 Oda Nobunaga began his conquest of Kai Province, staying at Iwamura Castle to receive reports of the results of the war.

Subsequent to Meiji revolution, all of the remaining buildings of the castle were demolished in 1873, with the exception of one gate which was moved to a neighboring temple.