[1][2] Initial fortification by the Virginia Company's (Bermuda was originally settled as an extension of Virginia) first deliberate settlers in 1612, was around the inlets to St. George's Harbour, but by the end of that year work had commenced on Castle Harbour's defences, starting at Castle Island (previously called Gurnett's Head, and King's Island).

Although neither struck, the Spanish vessel abandoned the attack (its crew did not realise that the gunners in the fort had only enough ammunition for one more shot).

This fort was used as late as the Second World War by Bermuda's military garrison, with soldiers living in tents within its walls, watching over the channels with machine guns.

Those then existing were depicted and described in the 1624 book, The Generall Historie of Virginia, New-England, and the Summer Isles, by Captain John Smith.

[5] As a result of their historical significance, they have been made a UNESCO World Heritage Site,[6] together with St. George's Town and other nearby fortifications, like Fort St. Catherine (UNESCO World Heritage Site: Historic Town of St George and Related Fortifications, Bermuda).