At the age of 17 he passed his Kriegsabitur (war time diploma, a school leaving certificate under accelerated conditions) and was assigned as Fahnenjunker (Cadet Officer) to the replacement battalion of 127th Infantry Regiment (9.

For his service during the War, he was decorated with both classes of the Prussian Iron Cross and also received Württemberg Military Merit Medal for bravery.

He later led 8th Company (Machine Gun) of his regiment until October 1929, when he was assigned to the staff of 5th Division under Generalleutnant Hans Freiherr Seutter von Lötzen in Stuttgart.

[2] Scherff was ordered back to Berlin in October 1938 and assigned to the Oberkommando des Heeres (OKH - supreme command of the Army) for service with the 7th Department of the Oberquartiermeister (Higher logistics general).

[2] Following the outbreak of World War II, Scherff assumed duty as Chief of the 7th Department in OKH and was promoted to Oberst (Colonel) in September 1941.

[2] Scherff was responsible for the destruction of parts of the complete stenographic record of Hitler's military conferences despite not having the authority to do so.

His personal copies were also "probably" burned, according to historians, as "Scherff made it plain that his opinion of Hitler as a general had changed, and he strongly criticized the military strategy of the last few years.