He also appeared in The Day the Earth Stood Still (1951) and Ben-Hur (1959), and is additionally known for his roles as the titular character in Gunga Din (1939) and as the "High Lama" in Lost Horizon (1937).

As a child, he appeared in Yiddish theatre productions with his mother, who after moving to the United States became a prominent actress and vaudeville star.

He also worked for several years as a teacher, and then dean, of mathematics at the Bronx Cultural Institute, a college preparatory school, before returning to acting in 1915.

[4] Jaffe began to work in film in 1934, rising to prominence with his first role as the mad Tsar Peter III in The Scarlet Empress.

Despite being blacklisted, he was hired first by Robert Wise for The Day the Earth Stood Still and then by director William Wyler for his role in the 1959 Academy Award-winning version of Ben-Hur.

He also appeared with an all-star cast in the TV pilot film of Rod Serling's Night Gallery and as Emperor Norton in one episode of Bonanza.