Nicknamed "the Gray Ghost", he played college football for the Oregon State Beavers and was a first-team All-Pacific Coast Conference (PCC) performer and second-team All-American.

[6] By his sophomore year, Gray had developed into a "sensational halfback", being described as "outstanding in every game" while being noted as a triple-threat man who could pass, run and punt.

[7] He tied for the scoring lead in the Pacific Coast Conference (PCC) and helped the Oregon State Beavers compile a record of 6–4–1.

[12] A reporter for the Corvallis Gazette-Times said it was "Not a bad name, either, as many opposing tacklers are willing to bet that Joe is just an apparition who can't be brought down.

[6] The Los Angeles Times described him as having a "heart of a true champion" and praised him for having "just about everything", noting that "He's a rugged, 60-minute player, kicks well, can run with the ball and plays a bang-up game on defense.

[13] Gray was selected in the first round of the 1938 NFL draft – 10th overall by the Chicago Bears – but declined their offer to work as an agriculture inspector, in part from a knee injury.

[3][4][13] He worked with the Oregon Department of Agriculture starting in 1938 and remained there until retiring in 1980, having risen to the post of administrator of the food and dairy division.