He conducted the church choir and started to write his first chamber music works that were transmitted on the radio Gliwice.
In 1929 Jary was accepted at the Staatliche Akademische Musikhochschule at Berlin, meanwhile he made money playing as a pianist at cafés or movies.
When Jary delivered his graduation concert on 8 February 1933, he was bullied by members of the Kampfbund für deutsche Kultur.
Paul Graener, the new director of the Stern’sches Konservatorium denigrated his concert as "the cultural bolshevistic musical stammering of a Polish Jew".
Forced to stay in Germany, he wrote, often with the lyricist Bruno Balz, songs for movies that became hits, including: Ich weiß, es wird einmal ein Wunder geschehen and Davon geht die Welt nicht unter.
In 1949 he moved to Hamburg, and Jary was very successful as film score followed film score including „Leise rauscht es am Missouri“, „Das machen nur die Beine von Dolores“, „Mäki-Boogie“, „Heut' liegt was in der Luft“ and others.
These songs were interpreted by artists such as Zarah Leander, Rosita Serrano, Evelyn Künneke, Gerhard Wendland, Heinz Rühmann, and Hans Albers.