During his first stay on the island (1809–1813) he had an affair with a Japanese woman and the couple had a child, who died in 1813.
When he arrived in Dejima for the second time in August 1817 he was accompanied by his wife Titia Bergsma, whom he had married in 1815; his son Johannes; Petronella Muns, a Dutch wetnurse; and an Indonesian maid.
In the short time they stayed there, till December 1817, they were often drawn by artists, who had never seen other than Japanese women, and 500 different prints widely circulated throughout the country.
Blomhoff is commemorated in the specific name of an Asiatic pit viper, Gloydius blomhoffii.
Blomhoff was succeeded as director of Dejima by Johan Willem de Stürler.