He played professionally in Hungary, Czechoslovakia, Austria and the United States, earning two caps, scoring two goals, with the Hungarian national team in 1922.
In November 1923, Makkabi played an exhibition game against SK Rapid Wien, crushing them 4-1 off two Schwarz goals.
Impressed by the high pay and relatively minor anti-Semitism compared to Europe, Schwarz and several of his teammates decided to move to the U.S. following the conclusion of the tour.
When that happened, Schwarz signed for Rangers F.C., but was unable to join the club due to labor restrictions in Great Britain.
After the Rangers deal fell through, Schwarz helped form New York Hakoah in the Eastern Professional Soccer League.
While the Americans defeated the St. Louis Shamrocks in the 1937 National Challenge Cup, Schwarz did not play in the final game as he had broken his leg in February 1937.
On 19 May 1935, Schwarz played an unofficial international match for the United States men's national soccer team in a 5–1 loss to Scotland.
That spring, the ASL collapsed and Schwarz was instrumental in the creation of a second American Soccer League which began playing in the fall of 1933.
Schwarz also worked during World War II to convince American GIs living in Britain to play soccer, in hopes of growing the sport in America.
He was also not set on the outdoor game, but in 1960, he also organized one of the first major indoor soccer tournaments at Madison Square Garden.