By then a national hero, Graf was withdrawn from combat operations and posted to a fighter pilot training school in France before being tasked with the setting up of a new special unit: Jagdgeschwader 50 (JG 50—Fighter Wing 50).

When I./JG 51 was briefly reequipped with the Czech-built Avia B-534 biplane in July 1939, it gave Graf an opportunity to prove his flying expertise as well as to restore his self-confidence.

[9] When Germany invaded Poland in September 1939, I./JG 51 was stationed by the French border at Speyer and Graf was promoted to Feldwebel (a higher grade of sergeant).

[9] During this period of the so-called "Phoney War", Graf flew 21 combat sorties without firing his guns and was still considered an unreliable pilot.

[9][5] At Merseburg, Graf met and befriended two other pilot trainees, Alfred Grislawski and Heinrich Füllgrabe,[10] with whom he would later spend much of his combat career.

[12] During that operation, Graf's unit was kept back to defend the strategically vital Ploiești oilfields from Allied bombing raids, which never eventuated.

[12] The Gruppe was not involved in most of the fighting in the opening phases of Operation Barbarossa as it was once again kept back to defend Ploiești and the Romanian ports from Soviet bombers.

[16] The pilots’ mood was not helped when Reichsmarschall Hermann Göring, the commander-in-chief of the Luftwaffe, criticised the unit on 4 July for its lack of success compared to virtually all the other Jagdgruppen.

[17][16] On 1 August, after German advances had removed the threat of Soviet air attacks on Romania, the unit was transferred to the front line and the major Luftwaffe airbase at Belaya Tserkov in Ukraine south of Kiev.

[18] Graf achieved his first victory in the early hours of 4 August when his squadron was escorting a Junkers Ju 87 dive-bombing strike, shooting down one of a pair of attacking Polikarpov I-16 fighters.

[21] As the German ground forces advanced across Ukraine in August and September, the air units of JG 52 kept pace – constantly moving on to forward airbases.

[42][37][44] It was at this time that his squadron got the famous moniker, the "Karaya Staffel", from a record that Süß would incessantly play at the airfield and hum along to in the air.

In mid-August, III./JG 52 moved forward to provide air cover as the army tried to establish bridgeheads across the Kuban River to capture the Black Sea ports.

[74] On 28 January 1943 Graf took command of Ergänzungs-Jagdgruppe Ost (Fighter Training Group East) [Note 9] based in occupied France.

The main base was at St. Jean d'Angély 70 miles (110 kilometers) north of Bordeaux on the Atlantic coast although Graf spent most of his time at the Toulouse-Blagnac Airport.

He transferred his old friends, Grislawski, Süss and Füllgrabe, from III./ JG 52, as well as a number of football players serving as administrators in his JGr Ost training unit.

[87][91] While in Berlin organizing the necessary transfers, Graf was introduced to the young film actress, Jola Jobst, whom he later wed.[87][93] The new assignment was then delayed for two months for political reasons.

Graf was to run one final pilot-training program: the latest draft of Spanish volunteers heading to the Eastern Front – the 4th Escudrilla Azul (4th Blue Squadron).

[Note 11] The unit was finally declared combat ready on 31 July 1943, albeit with only nineteen aircraft and made up of a single, three-squadron, Gruppe.

[108] En route from Jever, Graf and Mader were nearly shot down by a flight of two Mosquitos over the North Sea Coast, while flying in an unarmed Messerschmitt Bf 108 Taifun.

[115] Sunday 20 February marked the start of "Big Week" – six consecutive days of concentrated bombing by the USAAF designed to overwhelm the German defences.

[93] As the winter weather got worse, the lull allowed Graf to organise a football match between his Roten Jäger and a local Kraków team, watched by 20,000 German soldiers.

On New Year's Day, the Luftwaffe's western command launched Operation Bodenplatte, against Allied air forces based in the Low Countries.

In response, a meeting of senior Geschwader commanders, in early January (including Graf), agreed to approach Göring and demand major changes.

[123][124][125][Note 14] When confronted, Göring was furious and immediately demoted all the fighter leaders aside from Graf, who stayed as Kommodore of JG 52 because of the unit's exemplary war record.

[123] The major Soviet Vistula-Oder offensive began in January which saw a number of the fighter units transferred from Reich Defence to defend the Eastern Front.

[128] Instead, he led the 2,000 unit-personnel and fleeing local citizens on a march through Bohemia[133][76] to cross the Moldau River (the nominal Allied stop-line).

[136][134] The author, Musciano, mentions that he was subject to solitary confinement for extended periods, but that he had his final order hidden in his shoe and would re-read it to keep his sanity.

[136] In 1971 Graf made his own statement to the newspaper, "Bild am Sonntag", saying that he, along with others including Hartmann, had briefly joined the BDO (an anti-Nazi group of German ex-officer prisoners) as a way to survive the psychological deprivation of the imprisonment.

Bergström et al. say this is borne out by the Russian RGVA archive of Graf's POW file which makes no mention of extended co-operation with pro-Soviet groups(the BDO was disbanded after only a few months).