The island has long been occupied, due in part to its easily defended nature, and that it allows easy view of the sea access to Hyères and Toulon.

After the King confided Brégançon to Provençal captains, the current fort was built on the island in 1483 by Jean de Baudricourt as part of the French monarchy's coastal defence efforts.

In 1574, King Henry III of France donated Brégançon by letters patent to Antoine Escalin des Aymars, baron of the guard, captain general of the galleys.

Napoléon Bonaparte became interested in Brégançon after the recapture of Toulon, and after initial repairs endowed it with improved artillery, and then strengthened the garrison with a company of Imperial veterans.

Emmanuel Macron has received only a few foreign dignitaries, including the following: The fort served as El Supremo's fortress in the 1951 film Captain Horatio Hornblower.