Eltz-Rübenach was born in Wahn (today part of Cologne), a scion of the Rhenish noble house of Eltz.

He spent several years in the United States working from 1911 to 1914 as a technical expert in the German consulate in New York City.

At the outbreak of World War I he returned to Germany and worked in military rail transport.

He retained both these portfolios in the cabinets formed by Chancellor Kurt von Schleicher (3 December 1932) and, after the Nazi Machtergreifung, by Adolf Hitler (30 January 1933).

[3] Some time later, when his wife refused to accept the Cross of Honour of the German Mother, a Nazi decoration, he and his family became "suspect persons" and were placed under surveillance by the Gestapo.