He was born in Linden near Hanover, and studied Protestant Theology, Philosophy, and Oriental Languages in Halle and Göttingen, where he received a PhD in 1887.

His professional career developed in Göttingen, where he was Stiftsinspektor (from 1888), Privatdozent (from 1891), Extraordinarius (from 1914), and Professor for Old Testament (from 1919).

Influenced by his teacher Paul de Lagarde, Rahlfs's academic interest focused on the Septuagint, the Greek translation of the Hebrew Bible.

Together with Rudolf Smend and others, Rahlfs was responsible for the creation of the Septuaginta-Unternehmen under Göttingen's and Berlin's Academies of Sciences and Humanities in 1907, which he directed from 1908 until 1933.

Its goal has been to reconstruct the original wording of the Septuagint, and since Rahlfs' death it had published twenty volumes.