Along with Johann Adolph Hasse, he is considered to be the most important German composer of Italian opera of his time.

In 1714, he followed his brother, Johann Gottlieb Graun, to the school of the Kreuzkirche, Dresden, and sang in the Dresdner Kreuzchor and the chorus of the Opernhaus am Zwinger.

He studied singing with Christian Petzold and composition with Johann Christoph Schmidt [de] (1664–1728).

In 1735, Graun moved to Rheinsberg in Brandenburg, after he had written the opera Lo specchio della fedeltà for the marriage of the then crown prince Frederick (the Great) and Elisabeth Christine in Schloss Salzdahlum in 1733.

He was Kapellmeister to Frederick the Great from his ascension to the throne in 1740 until Graun's death nineteen years later in Berlin.