Born as the sixth child of a peasant family of Belarusian[2] ethnicity in the Mogilev Governorate of the Russian Empire (present-day Belarus), he was employed in their village and graduated from a rural school.

Commanding a tank company in the Red Army's September 1939 Polish campaign as part of the troops on the Belorussian front, he also served in the 1939–40 Winter War.

Yakubovsky entered the war in its early days on the western border as commander of a tank battalion, fighting heroically in the most difficult defensive battles in Belarus.

For his heroism at Fastiv, where his unit destroyed 30 enemy tanks in a single day, Yakubovsky was awarded the title Hero of the Soviet Union.

In April 1960 he was appointed Commander of the Group of Soviet Forces in Germany, being in post in the midst of the Berlin Crisis of 1961, when the threat of armed conflict in Europe dramatically escalated.

After stabilizing the situation in April 1962, Army General Yakubovsky again returned to the post of Commander of the Group of Soviet Forces in Germany.