Further, it was his opinion that, should an enemy attack from the sea, that defenses should be concentrated at the ports and anchorages, defended by the local population, under the responsibility of each municipal authority.

The final plan for Angra do Heroísmo and Terceira included elaboration of the coastal defenses at the Fort of São Sebastião (or the Castelinho as it is known), the Porto das Pipas and the Fort of São Sebastião (in the area of Ribeira Seca); the plan was later continued by Ciprião de Figueiredo e Vasconcelos, then governor of the islands, to encompass over fifty defensive works along the coast of the island.

With the start of Anglo-Spanish War (1585–1604), it was strengthened as privateers, such as Francis Drake (1587) and Robert Devereux, 2nd Earl of Essex, whose adjunct Walter Raleigh, attacked the neighbouring settlement in Horta, during the summer of 1597.

The new project served to two functions: The construction started in 1593, with the first cornerstone at the Santa Catarina bastion (northwest angle) in a ceremony presided by the Governor of the Presidio, António de la Puebla, and the Bishop of Angra, Manuel Gouveia.

[4] Later the engineers Girolamo Francès and Giovanni Battista Cairati (who was the royal architect in the court of Philip II of Spain) participated in this project, renaming the fortress the Fortaleza de São Filipe (in his monarch's honour).

At this time, the three interconnected cisterns (with a capacity of 750,000 kilograms), stables (on the site of the Governor's palace) and the chapel of Santa Catarina de Sena (which was later re-baptised in honour of the Holy Spirit).

The built environment was completed with local boulder stone and basalt by rocksmiths and masons, and a great part of the labourers were actually men condemned to the gallows.

At the outbreak of the First World War, Portugal was a neutral nation, but this progressively changed due in most part to its concerns over its dependencies and economic blockades with its main markets.

Meanwhile, it became a communications centre (Centro de Comunicações da Armada) for the Portuguese Armed Forces, and later a radio antenna was erected to create and support Pico Rádio.

The remaining fortress complex was classified on 18 August 1943 as a Property of Public Interest (under decree No.32-973) by the Direcção-Geral dos Edifícios e Monumentos Nacionais (DGEMN) (General-Directorate for Buildings and National Monuments).

[4] In 1956 the Serviços dos Monumentos Nacionais (National Monument Services) completed some renovations of the Church, while a recuperation of the fort began with a smaller conservation project in 1959.