[1][6] As the "head" of the Terra da Miranda, Algoso was an important fortress; a "nuclear point of support with the regal authority" in this peripheric region.

[1][7] Unfortunately, little remains of the early construction, although it assumed that the castle was built on the Romanesque principals of the time, that included a keep tower surrounded by a walled fortification and battlements.

[4] The town and municipality of Ylgoso, Ulgoso ou São Sebastião de Algoso, was part of the bishopric and district of Bragança.

These changes had an effect on the annual rents paid by their commander: between 7000 and 8000 cruzados, including products from the abbeys of Travanca, Sendim, Vilar Seco, Duas Igrejas and Guide.

[1] During this period the heptagonal keep tower was constructed, in order to better resist attacks and providing a more adequate availability of angles to inflict damage on aggressors.

[1][4] The commander of the Order, Father Pedro Lourenço, in 1291, appeared in a dispute between the King (then D. Dinis) and the Knights Hospitaller; in this dispute the Knights Hospitaller and Command of Ulgoso were united in their afront of the King, and included the villages of Serapicos, Vila Chã da Beira, Uva, Mora, Saldanha, Sendim, Picote, Vilar Seco, Vinhão and couples of Cerceo.

[4] In August 1341, a sentence from King D. Afonso IV informed that, in that year, Father D. Álvaro Gonçalves Pereira was the "Prior of Hospital in his kingdoms, that he pertained to the Castle of Ulgoso with its term since time in memorial".

On his accession to the throne of Portugal, Philip I favoured and conceded privileges on municipality of Algoso (1 January 1592), putting the castle authority and noble's allegiance into question.

In 1588, though, Friar Gonçalo de Azevedo, a Knight Hospitaller, and son of Diogo Almeida, was nominated by King Philip I, to act as alcaide and commander at the castle.

It is unclear of the state of the fortifications during the intervening years, such that by the 17th century, the castle was already abandoned,[1] and the settlement of Algoso took on a larger role in regional authority.

[4] The principal tower and cistern were cleared and renovated during this time, but further work on the walls was needed in the intervening years, that included the consolidation of various points with schist masonry and clay cement and sand.

[4] On 1 June 1992, the property became the responsibility of the Instituto Português do Património Arquitetónico (Portuguese Institute for Architectural Patrimony), by decree law 106F/92 (Diário da República, Série 1A, 126).

[4] The castle is situated in a rural countryside, isolated on a soft escarpment 690 metres (2,260 ft) above sea level, called the Cabeço da Penenciada overlooking the right margin of the Angueira River.

[4] The stone castle is designed in a rectangular layout, with corbel walls and a three-story square keep tower with a Gothic access doorway; the superior accessway in similar pointed arch and machicolations, is crowned by merlons with arrowslits.

[4] The remnants of the small rock castle and few lines of wall fortifications are interrupted by stones and cliff faces, approximately 12 metres (39 ft) tall.