[1][2][3] Born in Rouen around 1565, he moved to London in 1568 with his Huguenot parents Peter and Epiphany Oliver to escape the Wars of Religion in France.
He then studied miniature painting under Nicholas Hilliard; and developed a naturalistic style, which was largely influenced by Italian and Flemish art.
In 1602, he married Sara, daughter of the well-known portrait painter Marcus Gheeraerts the Elder (c. 1520 – c. 1590) and his wife Susannah de Critz.
After the death of Elizabeth I, he became a painter of James I's court, painting numerous portraits of the queen Anne of Denmark and Henry Frederick, Prince of Wales.
He accompanied Anne of Denmark on her progresses in Oxfordshire, Berkshire, and Kent in 1605, and was appointed "her Majesties painter in the art of lymning".