After the outbreak of WW2 Ollenhauer travelled across Europe in order to avoid Nazi persecution, first going to Denmark, then France, Spain, Portugal, and eventually London, where he remained until the end of the war.

In London, he kept close ties to the Labour Party, which financially supported the expatriate SPD (called SoPaDe), of which Ollenhauer was a member.

He ran as the SPD's candidate for Chancellor of Germany in the 1953 and 1957 German elections, both of which were lost to Konrad Adenauer's CDU.

In 1957, Ollenhauer called for a trans-European security alliance (in place of NATO and the Warsaw Pact), in which a reunified Germany would serve as an equal partner.

In 1961, Ollenhauer declined to run for Chancellor a third time and instead supported the candidacy of Berlin mayor Willy Brandt.