Carter began to use his fists defending himself on the streets of Harlem, but later received training at a Catholic Boys Club, making his amateur debut at the age of fourteen.

[1] On May 25, 1951, Carter took the World Lightweight Championship from reigning champion Ike Williams in a fourteenth-round TKO at Madison Square Garden.

[5] In his first title defense before 7,251 fans on November 14, 1951, he went up against Art Aragon at the Olympic Stadium in Los Angeles, winning in a fifteen-round unanimous decision.

[4] He lost the title on May 14, 1952, against Lauro Salas at the Olympic Auditorium in Los Angeles in a fifteen-round split decision that was a jarring upset for many.

[6] Carter took the World Lightweight Championship for a second time on October 15, 1952, against Mexican national Lauro Salas in a fifteen around unanimous decision at the Chicago Stadium before a small crowd of 5,283.

[8] He next fought George Araujo on June 12, 1953, in a lightweight world title bout at Madison Square Garden, winning in a thirteenth-round TKO.

[1] On March 5, 1954, Carter lost his second World Lightweight Championship against Paddy DeMarco in a fifteen-round unanimous decision at New York's Madison Square Garden.

[9] On November 17, 1954, Carter took the lightweight championship back from Paddy DeMarco in a fifteen-round TKO at the Cow Palace, in Daily City, California.