His Broadway credits included Once There Was a Russian (1961), Lily of the Valley (1942), Eight Bells (1933), Alien Corn (1933), Grand Hotel (1930), Half Gods (1929), and The Channel Road (1929).

His German accent and large stature kept him busy during World War II, playing sinister Nazi characters in a series of wartime thrillers.

During this period, he also appeared in several films by director Ernst Lubitsch, a fellow German émigré, including Ninotchka (1939), portraying a Russian, and in To Be or Not to Be (1942) as the pompous Nazi Colonel "Concentration-Camp Erhardt".

Ruman continued his trend of portraying over-the-top German characters later in his career for Lubitsch's protege Billy Wilder, in his films The Emperor Waltz (1948), and Stalag 17 (1953).

He guest-starred as pompous Broadway director Eric von Bissell in the memorable 1965 episode of The Addams Family, "My Fair Cousin Itt".