Sheppard began his career as a professional boxer on September 24, 1938, knocking out Larry White in the first round at Madison Square Garden.
In June of that year, he beat Q. Murray; then, on August 2, he lost by decision to future Hall of Famer Jimmy Bivins.
Sheppard won five of his next seven fights, until on July 27, 1942, he lost to future world Light Heavyweight champion Joey Maxim, by a ten-round decision.
This result, which would be the only knockout loss in Maxim's career, was a surprise to Ring Magazine writers and many boxing fans.
Only twenty-six days later, he went into the ring with a fighter who was 12-0 before their fight: Sheppard defeated Clint Conway by a ten-round decision on April 26.
On January 24, 1944, he had one of his best performances when he beat future challenger for the world heavyweight championship Gus Dorazio by a ten-round unanimous decision.
Sheppard's next fourteen fights were almost all against the elite of the Light Heavyweight division: He beat Buddy Walker by a knockout in eight rounds in a rematch.
Then he beat Tony Shucco by knockout in five, before losing to Q. Murray and to future world champion Melio Bettina, both by ten-round decisions.
Sheppard then had consecutive wins over Nate Bolden, Johnny Allen, and Perkins in a rematch, before facing Jersey Joe Walcott again, and Archie Moore.