The castle's history started in the Middle Ages when a chapel dedicated to Our Lady of Pena was built on the top of the hill above Sintra.

Nonetheless, the chapel (and its works of marble and alabaster attributed to Nicolau Chanterene) escaped without significant damage.

In 1838, as King consort Ferdinand II, he decided to acquire the old monastery, all of the surrounding lands, the nearby Castle of the Moors and a few other estates in the area.

King Ferdinand then set out to transform the remains of the monastery into a palace that would serve as a summer residence for the Portuguese royal family.

The commission for the Romantic style rebuilding was given to Lieutenant-General and mining engineer Wilhelm Ludwig von Eschwege.

The construction took place between 1842 and 1854, although it was almost completed in 1847: King Ferdinand and Queen Maria II intervened decisively on matters of decoration and symbolism.

Over time the colors of the red and yellow façades faded, and for many years the palace was visually identified as being entirely gray.

Structurally, it can be divided in four sections: As many elements as possible were preserved of the remains of the Hieronymite convent including the cloister, the dining room, the sacristy, and the Manueline-Renaissance chapel.

It has amazing stuccos, painted walls in trompe-l'œil and various revetments in tile from the 19th century, forming part of the numerous royal collections.