Zarubaev was educated at the Mikhailovsky Voronezh Cadet Corps and Second Constantine Military Academy and commissioned as an officer in the Imperial Guards Grenadier Regiment in 1868.

Likewise, at the Battle of Liaoyang, he successfully held his position against repeated Japanese attacks, but was again refused reinforcements by Kuropatkin and ordered to retreat.

At the Battle of Shaho, his forces were largely held in reserve, and were used to reinforce the line covering Kuropatkin's retreat.

However, this combined force was cancelled on 25 February, and after Kuropatkin was replaced by General Nikolai Linevich, Zarubaev was made temporary commander of the Russian First Manchurian Army.

From 1906-1909, he served as Inspector-General of the Infantry and chaired a committee tasked to improve the personnel and officer corps of the army.