After service in World War I, he was active with Max Reinhardt's Theater in der Josefstadt in Vienna, followed by engagements at various theaters in Munich and Vienna, and then to the Berlin stage where he had a popular following playing singing and dancing roles, including as Prinz Orlofsky in Die Fledermaus.

With Hitler's rise to power in 1933, Karlweis, who was Jewish, returned to the Austrian stage; but with the Nazi Anschluss in 1938 he was again forced to flee, first to Switzerland and then Paris where, at both locations, he took to the stage with fellow Austrian law school dropout, writer/director, and cabaret performer Karl Farkas.

When German forces took Paris in 1940, Karlweis, along with his friend Austrian writer Friedrich Torberg, fled via Spain to Portugal, whence they made their way to New York.

Not knowing English, Karlweis was at first limited to acting in German emmigré productions, but he soon got his English legs and appeared in eight Broadway shows in the early 1940s,[1] including the title role in Franz Werfel's long-running "Jacobowsky and the Colonel," garnering outstanding reviews.

In 1946, Karlweis married theatrical and film producer Ninon Tallon, the niece of three-time prime minister of France Édouard Herriot.