It was a fortification of modest proportions, established on a hilltop (the current Alto da Memória) with a dominant position over the valley, river and harbour.

[3] Similarly, Gaspar Frutuoso mentioned these mills and waterways, noting that: in the hillock that circles Angra, in one of the higher hilltops to the north, is a castle fort with munitions and artillery, newly renovated and provisioned, which had earlier been weak, erected solely for the shelter and defense of its residents.

For this reason, in 1482, the Infanta Beatrice, who governed the islands on behalf of her son (the Captain-Major D. Diogo), sent a letter to Álvaro Martins, warning him that Castilian vessels continued to haunt the seas of the Azores.

[1] After completing his task of building the Castle of São Luís (around 1493), Anes Rebelo wed a niece of João Vaz Corte-Real's spouse.

On 20 May 1844, work began to demolish the old walls, to construct a memorial in honour former-King Pedro IV of Portugal who was instrumental in the restoration of the Liberal regime of his daughter Queen Maria II, and whose regency was based in Terceira.

At the time of its inauguration (in 1856), the municipal council deliberated whether to name the newly built courtyard the Praça D. Pedro IV, but popular sentiment opted for the name Alto da Memória (Heights of Memory).

The Castle of Moinhos is situated on the hilltop plateau that separates the eastern portion of the civil parish of Sé from Nossa Senhora da Conceição.