When Zaamslag Castle was built in the 12th century, its location was part of the County of Flanders.

On some of these ridges near Ghent's north and northeast in the year 1000, a population began to grow.

[2] Between 1584 and 1586 all the dikes in the area were destroyed by the Dutch Republic in order to keep control of the Scheldt.

This did not immediately destroy Zaamslag village and its castle, because both were above Amsterdam Ordnance Datum.

Its diameter is about 48 m. Around most of the Torenberg is a depression at about 0.7 m above AOD, which can be identified as the moat, which is in turn about 0.5 m below the surrounding field.

[8][9] This mistake did not matter much, because in both cases, it could be assumed that the main castle or Motte started as a wooden tower on an artificial hill.

[8] During the second half of the 13th century, the level of the outer bailey was raised by about 0.75 m. The bridgehead of the village bridge on the island side was rebuilt in brick.

Instead of the bridge, a double wall was constructed, that kept the water of the moat out of the entrance to the outer bailey.

[11] An eyewitness account of the tower on the motte was given by Jona van Middelhoven after its destruction.

In the mid-15th century, it was heavily damaged during the war between Philip the Good and Ghent.

In 1587 the government of Zeeland gave official permission to loot brick at: vande kercke van Saeftinghe, Samslach, thuijs van Bouweloo ende het Tempel huijs..[15] While it's not clear whether this refers to the castle, instead of the church of Zaamslag, it reflected a trend.

(thuys van Zaemslacht) In 1593 the Zeeland government ordered that the tower of Zaamslag should not be destroyed.

[17] After the lands around Zaamslag had been reclaimed, a farm with a large barn and carriage house was erected.

It was supposed that the buildings on the outer bailey protected that area, but this proved a mistake.

[18] When in 1983 a storm inflicted heavy damage on the monumental barn and carriage house, these were demolished.

[18] Protests by the local history society, and more importantly, the Rijksdienst voor het Oudheidkundig Bodemonderzoek (ROB, National Archaeological Service) could not to rescue the terrain.

[19] In the Summer of 1987 the National Archaeological Service was allowed to make some hasty excavations.

In 2014 a depiction of the castle entrance dreefmuren was placed on the corner of Plein and Voorburch streets.