From 1927 to 1934 he was the constant accompanist of the famous German baritone Heinrich Schlusnus, but he fell out with him when the singer made a career under the National Socialists.

[2] But he accompanied the outstanding Austrian violinist and composer Fritz Kreisler, with whom he went to tour South America in 1935.

Rupp also accompanied singers Lotte Lehmann, Sigrid Onégin, Maria Stader and Beniamino Gigli, and was a highly esteemed chamber musician who, among others, performed with cellist Emanuel Feuermann[3] and violist William Primrose.

In 1938 he moved to New York and soon became the permanent accompanist of contralto Marian Anderson, until her retirement from the stage in 1965.

His last public performance took place at the Lockenhaus Festival in Austria in 1985 when he accompanied violist Rivka Golani.