Peter II, Count of Saint-Pol (1440–1482) was succeeded by his daughter, Françoise of Luxembourg, who married Philip of Cleves, Lord of Ravenstein (1459–1528).
When Françoise died childless in 1523, the castle and gardens went to her sister, Marie of Luxembourg (1472–1527), who was married to a French prince, Francis, Count of Vendôme.
[1] When Marie passed away in 1546, her grandson, Antoine de Bourbon (1518–1562), King of Navarre and Duke of Vendôme, inherited the estate.
[1][2] During a visit in 1671, Anne Marie Louise d'Orléans, Duchess of Montpensier was very impressed by the gardens as she wrote in her diary.
[1][2] She was accompanied by the French king Louis XIV, who summoned André Le Nôtre as well.
[1][2] Engravings made by Romeyn de Hooghe around 1680 show how the gardens appeared at that time.
[2] There were fountains, pavilions, triumphal arches, statues, mountains, an orangery, a theatre, artificial caves and other follies.
In the centre of the gardens a seven-sided pavilion was built, which served both as a temple of Hercules and an observatory.
[1] During a hunting party in 1775, Louis Engelbert, 6th Duke of Arenberg (1750–1820) was hit in the face by a shotgun and remained blind for the rest of his life.
Between 1780 and 1782, the French architect Charles de Wailly designed a sumptuous palace to replace the old castle.
[3] De Wailly also designed follies in the gardens, such as Roman ruins, a new herculanum and a temple dedicated to Apollo.
Prosper Louis, 7th Duke of Arenberg (1785–1861), a great lover of botany and horticulture, created glasshouses with more than 1,700 species.