After serving in the U.S. Army during World War II, he attended the University of Kentucky and continued to play football under head coach Bear Bryant starting in 1946.

Phelps was suspended for most of the 1948 season for breaking team rules, but he returned in 1949, when Kentucky played in the Orange Bowl.

[1] Playing football under head coach Bear Bryant, he led the nation in kickoff and punt return yards as a freshman in 1946.

[3] After graduating from Kentucky in 1949, Phelps was selected by the Cleveland Browns of the National Football League (NFL) in the sixth round of the 1950 draft.

[4] Led by quarterback Otto Graham and fullback Marion Motley, the Browns finished the 1950 season with a 10–2 record and won the NFL championship over the Los Angeles Rams.