Jacob's grandfather Maarten Ruychaver [Wikidata] had been a member of the council of the city of Haarlem between 1602 and 1618.

[2] Ruychaver resigned from office in November 1645 and travelled via São Tomé and Brazil back to the Dutch Republic.

Between 1645 and 1648, the Haarlem painter Frans Hals painted a portrait featuring what seems to be a Dutch family with two children and a servant of African descent.

According to Ineke Mok and Dineke Stam, it is likely that the family portrayed is that of Jacob Ruychaver and his wife Maria Hendrixs.

It was common for former directors-general of the Gold Coast to commission a painting with an African servant (compare Ruychaver's successor Jan Valckenburgh).