He was the son of Hans Christoph von Gagern, minister of state in Nassau; he attended the gymnasiums at Kreuznach, Mannheim, and Weilburg, and studied law from 1826 at Heidelberg, Utrecht, and Göttingen.
In 1833 he retired from the service of the Netherlands, married Franzina Lambert, of The Hague, and took up historical studies in order to fit himself for the position of Privatdozent at Bonn University.
Acquaintance with Catholics and with the historian Georg Friedrich von Böhmer [de], who was friendly to Catholicism, awakened in him respect and veneration for the Church.
The chief sources of his Catholic knowledge were, as he himself says, the Imitation of Christ by Thomas à Kempis, the study of Johann Möhler's Symbolik, and the New Testament.
When the King of Prussia declined the imperial crown offered to him and the Parliament of Frankfurt approached dissolution, Von Gagern and his party withdrew from the assembly.
In 1881, eight years after his retirement on a pension, Emperor Franz Joseph I made him a life member of the upper house of the imperial Austrian Parliament.