Born in Kleve and descended from a family of old German nobility [de], Johann was the son of Baron Kaspar von Elmpt (died 1730), and after receiving his initial education in his native town, he entered the service of the French Army.

In 1790 Elmpt was rewarded with the dignity of a Reichsgraf (Imperial Count) of the Holy Roman Empire, an honor that was confirmed in the Russian peerage in December of that same year.

He was given command of all army forces stationed in the Livonian Military Division based out of Riga, his rank was adjusted to that of General of Infantry, and he was given the honorary position of Chief of the prestigious Saint Petersburg Grenadier Regiment, rechristened under his own name.

On the day of Paul's formal coronation (5 April 1797), Elmpt was awarded the dual high honors of the Order of St. Andrew and promotion to the rank of Field Marshal.

On 10 January 1798 Field Marshal Elmpt was discharged from military service due to his advanced age, but was permitted the privilege to continue to wear his army uniform in public.