Due to its excellent location in the Veluwe, the castle used as hunting lodge by the dukes of Guelders and the princes of Orange.
[1][2] In 1516, Charles II, Duke of Guelders (1467-1538) sells the castle to his huntsman, Gerrit van Scherpenzeel, landdrost of the Veluwe.
[1][2] In his first part of his reign as stadtholder, he spent as much as ten weeks a year hunting in the Veluwe.
[1][2] William III and Mary Stuart (1662-1694) had various houses in this area, like the Hof te Dieren and the palace Het Loo.
[1][2][6] The Dutch national archives contain the design for the impressive gardens around the palace including various large ponds.
[1][2][6] Some scholars attribute the plan to the architect Daniel Marot (1661-1752)[1][2][6] Other scholars think that the design has been made by Jan van Arnhem (1636-1716), friend of William III and owner of Rozendael castle, where he designed similar gardens including a similar set of large ponds.
[1][2] Some scholars argue that instead of fire, the old castle was demolished and no replacement was constructed due to the early passing of the king.