It stands out over the older part of the town because of its imposing location over a higher area and because of the contrast between its dark walls and the prevailing whitewashed houses.
The earliest historical references date from the reign of Dom Afonso III of Portugal, the king who conceded the first charter to Portalegre in 1259, which gave it the status of vila (town) and made it the capital of a new autonomous municipality.
Starting in 1290, Afonso III's son, the king D. Dinis, reinforced the castle and built the city walls, motivated by the increasing need of defending the frontier and the increased importance of the town.
In course of the Portuguese Restoration War, the defences of the town were renewed and reinforced again, between 1641 e 1646.
In the Peninsular War there were combats in 1808 against the Napoleonic troops stationed in Portalegre, an event that may have triggered that the French general Louis Henri Loison imposed a heavy tribute to the town.