David Cordingly relates that Willem the Elder fathered two children out of wedlock in 1653, one “by his maidservant, and the other by her friend.

The immediate cause of the dispute was his affair with a married woman.” Michael S. Robinson noted that “on 17/27 July 1662, he and his wife agreed to part.

A condition of the separation was that the Elder could recover from his son Adriaen ‘two royal gifts’, presumably gifts from Charles II for work done in England.” Cordingly’s account further relates that the dispute was still continuing after another ten years, since “in the autumn of 1672 Judith complained to the woman’s husband.” Robinson adds that by 1674 the couple “must have been reconciled”, for at a chance meeting with Pieter Blaeu in Amsterdam in July the Elder explained that he was only visiting for a few days “in order to fetch his wife”.

Following the accession of William III and Mary II as King and Queen of England, it appears that this facility was no longer provided, and by 1691 he was living in Sackville Street, now close to Piccadilly Circus.

He died in London, and was buried in St James's Church, Piccadilly, at the south end of the street.