He squared off against many different Hall of Famers including the likes of Sugar Ray Robinson, Ike Williams, Beau Jack, Fritzie Zivic, Midget Wolgast, and Wesley Ramey.

Fighting as a flyweight, he lost a close decision in the finals to Pat Palmer of England, but managed to win the silver medal.

[4] On July 6, 1937, Berger topped Midget Wolgast in an eighth round points decision before a crowd of 3,000 at New York's Coney Island Velodrome.

[6] On September 9, 1937, at 133 1/4 pounds, Berger took the Canadian Lightweight championship against Dave Castilloux in a twelve-round points decision at the Forum in Montreal.

[11] On September 26, 1939, Berger won over Leonard Del Genio in a crowd pleasing eight round points decision at the New York Coliseum in the Bronx.

He came out aggressively in the first round opening cuts above Beahhuld's eyes and avenging a loss he took to Beauhuld on February 7, 1938, when Beahuld scored an eight-round points decision against him in St. Nicholas Arena.

[1] On February 20, 1942, Berger lost to Sugar Ray Robinson before 12,000 excited fans in a second-round TKO at Madison Square Garden.

[14] It was unfortunate for his legacy that one of his most memorable and well attended bouts for American fans was a loss, even if it was to one of the greatest boxers of all time with one of the longest winning streaks in history.

On February 15, 1944, Berger lost to Beau Jack in a ten-round unanimous points decision at Public Hall in Cleveland, Ohio.

[15] On January 22, 1945, Berger lost to talented black boxer Ike Williams in a fourth-round knockout, 2:51 into the round, at the Arena in Philadelphia, Pennsylvania before a crowd of 5,451.

In a 1972 interview with the Montreal Gazette, Berger claimed he was offered $10,000 by professional gamblers shortly after his last bout to throw a fight with Johnny Greco but refused.