He later passed on to his son, Magnus Gabriel De la Gardie, from whom it was purchased in 1669 by Queen Hedvig Eleonora of Sweden.

Hedvig Eleonora had grand plans for the palace and renamed it in 1684 Ulriksdal in honor of its intended future owner, her grandson Prince Ulric.

The prince, however, died at the age of one and Hedvig Eleonora kept the palace until her death in 1715 when the property was transferred to the crown for King Frederick I's disposal.

Several drawings by Nicodemus Tessin the Elder show a stately palace, three storeys high, with a lantern roof, furnished attic, and side wings extending the lakeside façade.

When building work eventually resumed by King Frederick I in the 1720s, the palace architect Carl Hårleman had different ideas than Tessin the Elder.

With the aid of architect Fredrik Wilhelm Scholander and through extensive purchases of antiques, Prince Charles was able to design and furnish the palace at his own taste.

The present chapel was designed by architect Fredrik Wilhelm Scholander and was built in 1864–1865 in the palace garden, in Dutch new Renaissance style with a certain influences from Venice.