Situated in a bay on a stretch of coast that is relatively difficult to access by sea or land, Ksar es-Seghir never grew in size to rival the other north Moroccan ports.

However, its sheltered position made it attractive as a military landing ground, a place for the safe and orderly embarkation and disembarkation of sea-borne troops, with little danger of disruption or molestation by enemy action.

In 1458, a Portuguese expeditionary force of 25,000 men and 200 ships led by King Afonso V of Portugal, assaulted and captured the town after a two-day battle on 23–24 October.

[6] The Marinid sultan Abd al-Haqq II of Morocco attempted to recover it immediately, laying sieges in late 1458 and again in the summer of 1459, to no avail.

Finding Portuguese holdings in Morocco expensive to maintain, King John III of Portugal decided to abandon it in 1533,[2] although the final evacuation of Ksar es-Seghir would be delayed until 1549.

It was recovered by Morocco thereafter, but the departing Portuguese had taken the trouble to evacuate the population, dismantle much of the fortifications and town, and dump debris and sand into the harbor, diminishing its immediate usefulness.

[6] Having lost its role as a transit port, Ksar es-Seghir collapsed in size and importance thereafter, becoming a relatively insignificant fishing town, amid the ruins of the old Moroccan citadel and Portuguese fort.

Ksar Sghir got a new lease on life in the 21st century, when it was slated as the site of a new naval base for the Royal Moroccan Navy (construction begun 2008, operational 2010).

[7][8] In 2007, a new commercial cargo port, Tanger-Med began being built nearby, around twelve kilometers to the northeast of Ksar Sghir.