When the war with Germany broke out in 1941, Sharokin served in the General Staff: first as Deputy Chief of the Operations Directorate, then Deputy Chief of the General Staff.
In August 1943, he received command of the 37th Army on the Steppe, 2nd and 3rd Ukrainian Fronts, which took part in the Poltava-Kremenchug, Dnieper, Nikopol–Krivoi Rog, Bereznegovato-Snigirevka, Odessa, Jassy-Kishinev and Bulgarian offensive operations.
In these battles, his 37th Army crossed the Dnieper, Southern Bug and Dniester rivers, and by the end of September 1944, reached the Bulgarian cities of Kazanlak, Yambol and Burgas.
With the 57th Army, he advanced through Hungary, Croatia and Austria and participated in the Budapest, Lake Balaton defensive, Nagykanizsa–Körmend, Vienna and Graz-Amstetten offensive operations.
By the decree of the Presidium of the Supreme Soviet of the USSR of April 28, 1945, Colonel-General Mikhail Nikolayevich Sharokhin was awarded the title of Hero of the Soviet Union with the Order of Lenin and the Gold Star medal for exemplary performance of command assignments on the front of the struggle against the German invaders and for the courage and heroism shown at the same time.