A former student of ÉNA, he was a junior minister (Secrétaire d'État) in the government of Raymond Barre, President of the regional council of Upper Normandy, and also a deputy.

Rufenacht's political career began in the then Gaullist party, the Union of Democrats for the Republic (UDR), and he remained in its successor parties, the Rally for the Republic (RPR), the UMP, and most recently, from 2015, the Republicans.

In the presidential election of 1981, he backed the former Gaullist prime minister Michel Debré for the presidency against his own party leader, Jacques Chirac, but supported him in 1988 and 1995 and served successfully as his campaign manager for the presidential election of 2002.

With Jacques Barrot, he was seen as one of several possible successors of Dominique de Villepin as Prime Minister of France.

In October 2010, he resigned from his post as mayor of Le Havre to be replaced by Édouard Philippe.