Herrmann served as Secretary of State in the Bavarian Labour Ministry from 1998 to 1999 and faction leader of the CSU in the Landtag of Bavaria from 2003 to 2007.

In October 2008 he was a candidate within the CSU party for the Minister-President of Bavaria but he withdrew candidacy in favor of Horst Seehofer.

Herrmann served as a CSU delegate to the Federal Convention for the purpose of electing the President of Germany in 2004,[2] 2009, 2010 and 2012.

In the negotiations to form a coalition government of the Christian Democrats and the Free Democratic Party (FDP) following the 2009 federal elections, Herrmann was part of the CDU/CSU delegation in the working group on internal and legal affairs, led by Wolfgang Schäuble and Sabine Leutheusser-Schnarrenberger; and of the working group on transport and building policies, led by Hans-Peter Friedrich and Patrick Döring.

After the 2017 constitutional referendum that handed new powers to Turkish President Recep Tayyip Erdoğan, he called for the European Union to end discussions with Turkey about EU membership and argued failing to issue consequences after the referendum vote would hurt Europe.