He performed mandatory military service as a one-year volunteer between 1908 and 1909 with the 13th (Hohenzollern) Foot Artillery Regiment, headquartered in Ulm.

A conservative German nationalist, with an antisemitic character, his radicalized war experience and sense of post-war social outrage led him to embrace extreme right-wing politics.

He co-founded the local chapter of the Nazi Party in Schwäbisch Hall in 1922, and became heavily engaged in propaganda activities as a public speaker.

In May 1924, he was elected as a member of the Völkisch-Social Bloc (VSB) electoral alliance to a seat in the Württemberg Landtag (state parliament), which he would hold until 1928, and again from 1929 until the dissolution of that body by the Nazis in October 1933.

However, on 5 May, Hitler elevated Murr to the newly created position of Württemberg Reichsstatthalter (Reich Governor), and Mergenthaler succeeded him as head of the cabinet with the new title of Ministerpräsident, while also retaining the portfolio of Culture Minister.

[5] Mergenthaler, since 1927, was also a long-serving member in the Party's paramilitary organization, the Sturmabteilung (SA), and he attained the rank of SA-Obergruppenführer in November 1938.

Clearly overshadowed by Murr, who held the highest Party and governmental posts, Mergenthaler nonetheless remained influential in his position as culture minister.

Mergenthaler intervened in parochial schools and banned teaching of parts of the Bible that he thought contrary to the "moral sense of the Germanic race", cut State contributions to the churches, forbade pastors who had not pledged allegiance to Hitler and, in 1939, finally ordered the introduction of a Nazi-tinged "Intuitive World Curriculum" in place of all religious education.

At the local level, his actions led to bitter conflicts between the Church, the Nazi Party and the school bureaucracy which alienated the devout population of Württemberg.