In the year 1618 Volkert Overlander - knight, mayor and advisor of the city of Amsterdam - bought the Lordship from the Count of Egmond.

[1] After Overlanders death Frans Banning Cocq, who had married his daughter Maria Overlander van Purmerland, inherited his influence and properties in the north of Amsterdam as well as the title Free Lord of Purmerland and Ilpendam.

After Banning Cocqs death in 1655 the castle belonged to the Amsterdam regent family De Graeff.

[2] Joost van den Vondel wrote poems about Pieter de Graeffs and Jacoba Bickers wedding in 1662 at castle Ilpenstein.

[3] The last residents of the castle were Christina Elisabeth de Graeff and her husband Jacob Gerrit van Garderen.