After completing vocational education with the Reichsbahn, Mittag served in a flak regiment of the Wehrmacht in the Second World War.

He joined the Communist Party of Germany (KPD) in 1945, became a member of the SED in 1946 and by 1958, when he had earned his doctorate with a dissertation entitled "Die Überlegenheit der sozialistischen Organisation und Leitung im Eisenbahnwesen der DDR gegenüber dem kapitalistischen Eisenbahnwesen" (en: The Superiority of Socialist Organisation and Performance in the Railroads of the GDR to the Capitalist Railroads), he became Secretary of the Economic Commission at the Politbüro.

He and Erich Apel designed the New System for Economic Management and Planning (NÖSPL), to modernise and streamline the formerly-bureaucratic economy of the GDR.

His leadership style was controversial, involving confrontations with ministers and demands for the summary dismissal of officials.

He was particularly close to Franz Josef Strauss, and in the early 1980s arranged the so-called "Billion Loan" from West Germany.