He was soon appointed to a job more suited to his specialized skills, as an officer of the Monuments, Fine Arts and Archives Section (MFAA) in Normandy and Paris, and, later, in Germany.

While in Paris, Rorimer had an auspicious encounter with Rose Valland, an art historian and employee of the Galerie Nationale du Jeu de Paume.

During the German occupation of France, the Reichsleiter Rosenberg Taskforce—the Nazi organization in charge of the systematic looting of art and cultural artifacts—designated the Jeu de Paume museum as the central location for storing and sorting their stolen treasures.

[8] While in Germany, Rorimer was promoted to chief of the MFAA Section of the Seventh Army Western Military District, where he was tasked to uncover and preserve significant works of art stolen by the Nazis.

[9] As director of the Met, Rorimer proved a capable administrator, though his directorship was plagued by contentious battles with trustees and the museum's staff.

[12][7][13] Rorimer was an inspiration for the character of James Granger, portrayed by Matt Damon in the George Clooney-directed film The Monuments Men, released in February 2014.