The castle was part of the defensive line controlled by the Knights Templar, a stronghold used during the Portuguese Reconquista, and adapted during the Restoration War to defend the frontier.

Within the castle fortification are two epigraphic inscriptions: one, the first to record a settlement established by a Portuguese monarch, while the second carving identifies the alterations occurring under the reign of King Denis.

[1] On 1 July 1279, the replacement or construction of the castle gate began: it was one of the first public works begun under the reign of King Denis (from an inscription located on the site).

[1] This project was part of the larger expansion of Portuguese control, conferred officially on 1 May 1282, when the King issued an expanded foral decree (charter) for Castro Marim and its surroundings.

[1] Designs were executed in 1509-1510, by Duarte de Armas, with several modernized barbicans (a missing or destroyed merlons) and irregular plant, that accompanied the topography of the region and encompassed the village.

In the eastern corner was a rectangular tower lined with merlons and accessed by a large gate with arched door; to the west was a smaller tower, but with a gate surmounted by a sculpted coat-of-arms (with main shield and five smaller shields); in the north, was a small arch close to the ground for sewage, flanked by barbicans; and to the north, the rectangular castle was constructed with cylindrical towers in each corner, surmounted by merlons and crowned by a conical wood structure (likely ceiling beams with tile); the southern aspect also included a cell block/tower, with low rectangular bulwark and two levels of windows.

[1] The damage extended to the Church of Santiago, which was completely ruined, resulting in its transfer to the Hermitage of Nossa Senhora dos Mártires,[6] while the castle was ordered reconstructed by King Joseph.

[5] The outer fortification of the city is in the form of an irregular triangle, encircled by a battlement with merlons, on the south by a platform, and two low rectangular towers on the east and west (both covered by terraces with simple doors).

The commemorative inscription marked the settlement of Castro Marim by King Afonso III was carved onto sandstone, with the coat-of-arms sculpted in relief, showing five castles, four shields forming a cross.

The 1274 carvings are well worn, with the Caroline-Gothic inscription stating: "ERA M CCC XII POPULAVIT CASTRUM MARINUS REX ALFONSUS PORTUGAL ET ALGARBI ET STI...".