His parents, Cornelia and Johann, operated an inn called "Zur weissen Schlangen" (At the White Snakes) in Frankfurt.
From 1759 to 1762, he worked with Johann Heinrich Tischbein at his studio in Kassel, which was attached to the court of Frederick II, Landgrave of Hesse-Kassel.
It was in Erfurt in 1773 that he met Christoph Martin Wieland and Friedrich Justin Bertuch, who introduced him at the court in Weimar.
In 1776, when Karl August took over the government, his first two acts were inviting Goethe to come there, and establishing what would become known as the "Princely Free Drawing School".
In 1786, he and Bertuch began publishing the Journal des Luxus und der Moden, generally considered to be one of the world's first fashion magazines.