Persoff is best known for roles as Leo in The Harder They Fall (1956), as Little Bonaparte in Some Like It Hot (1959), as Jake "Greasy Thumb" Guzik in The Untouchables (1959–1963), as Rebbe Mendel in Yentl (1983), and as the voice of Papa Mousekewitz in the animated film An American Tail (1986) and its sequels.

By 1929, he had become financially secure in New York City and sponsored the immigration of his wife and three sons (Boaz, Avraham, and Nehemiah himself) and two daughters (Tamar and Geula).

[9] In 1942, he was drafted into the United States Army and served for the remainder of World War II; he was assigned to the Special Services for entertainment.

In 1947, he was accepted into the Actors Studio and was one of the 26 members of the beginners' class taught by Elia Kazan, along with Martin Balsam, Julie Harris, Kim Hunter, Cloris Leachman, and James Whitmore.

[13] Among his early film roles was as the taxi driver during Marlon Brando's "I coulda been a contender" scene in On the Waterfront (1954);[2] Leo the accountant in The Harder They Fall (1956), with Humphrey Bogart and Rod Steiger;[2] and the gangster boss Little Bonaparte (a parody of Rico/Little Caesar) in Billy Wilder's film Some Like It Hot (1959),[14] and he appeared again with Steiger in Al Capone (also 1959).

[16] Persoff, alongside Dean Stockwell and Andrew Bloch, appeared as Darius the Great's wicked Babylonian administrators in the 1978 Greatest Heroes of the Bible episode Daniel in the Lions' Den.

[2] On April 5, 2022, at the age of 102, Persoff died of heart failure at a rehabilitation facility in San Luis Obispo, California.