Some of his better known performances include his roles as Professor Oddley in Ball of Fire (1941), Roger in No Time for Love (1943), Thomas Rogers in And Then There Were None (1945), Emperor Franz Joseph in The Emperor Waltz (1948), the Caterpillar in Alice in Wonderland (1951), Baron Popoff in The Merry Widow (1952), William Brown in Mutiny on the Bounty (1962), and Max Detweiler in The Sound of Music (1965).

After working as a music hall entertainer and overseer of a Jamaican banana plantation, he joined a touring British theatre troupe,[1] and then moved into television and film.

[3] Haydn was known for playing eccentric characters, such as Edwin Carp, Claud Curdle (Mr. Music, 1950), Richard Rancyd (Miss Tatlock's Millions, 1948) and Stanley Stayle (Dear Wife, 1949).

Notable performances included the voice of The Caterpillar in the 1951 Disney animated adaptation of Alice in Wonderland, and his small role of Herr Falkstein in the 1974 Mel Brooks comedy Young Frankenstein.

On April 1, 1964, he reprised his character of Edwin Carp in an episode of The Dick Van Dyke Show which saluted several old-time radio performers.