The core area of the castle is relatively small, measuring roughly 200 meters square, but unusually for the time, most of the walls on all of the enclosures are faced with stone.

During the Muromachi period, the area of Matsuzawa was denominated by the Kitabatake clan, who were strong supporters of Emperor Go-Daigo and the Southern Court.

After the assassination of Oda Nobunaga in 1582, Ise was invaded by the army of Toyotomi Hideyoshi, who installed his general Gamō Ujisato at Matsugashima as daimyō of a 123,000 koku domain.

The castle survived an attack by the forces of Ishida Mitsunari shortly before the decisive Battle of Sekigahara, and Furuta clan was re-confirmed in itsholdings and its kokudaka increased to 54,000 koku by the Tokugawa shogunate.

Despite the official policy of “one country-one castle”, the existing fortifications at Matsusaka were not destroyed, but were retained as an administrative center for the domain’s 179,000 koku holdings in southern Ise Province.

In 1877, a fire destroyed the palace within the second bailey, and in 1881 all of the remaining castle buildings were pulled down, with the exception of a single rice warehouse.

Although few structures now remain of the original castle, the site is open to the public as a park and the stone walls are in good preservation.