[3] According to a legend, a local church (Chiesa dei Novelli Innocenti) was founded in the early 14th century in honour of hundreds of participants to the so-called Children's Crusade of 1212 who perished in a shipwreck just off the island on their way to North Africa during a gale.

[citation needed] Carloforte was founded in the 18th century by around 30 families of coral fishers, originally from the Ligurian town of Pegli, near Genoa.

They asked the King of Piedmont-Sardinia Charles Emmanuel III for permission to settle down on the once uninhabited San Pietro Island instead.

When he granted them permission, the island was colonized (1739); the name Carloforte ("Charles the Strong", but also the "Carlo's Fort") was given to the town they then proceeded to found, in the Piedmontese king's honour.

To the north of the town (at the La Punta locale) is a collection of buildings that housed the former communally owned tuna-processing plant linked to the once-vibrant tuna fishing tonnare industry on the island.