Giovanni was close to Giuliano de' Medici, Duke of Nemours.
They were both members of the Compagnia della Cazzuola; a society that put on theatrical plays.
Gaddi inherited a collection of illuminated manuscripts and codices from his grandfather.
He was associated with many writers, particularly his secretary Annibal Caro, and appeared in Benvenuto Cellini's autobiography.
After the Sack of Rome (1527) he became one of Sebastiano del Piombo’s close friends.