He played college football for the Notre Dame Fighting Irish, winning the Heisman Trophy and Maxwell Award in 1949.

Also, he is one of four players, along with Angelo Bertelli, Cam Newton, and Joe Burrow to win the Heisman Trophy, a national championship, and be the first overall pick in the NFL Draft all in the same one-year span.

He also spearheaded an initiative for the inclusion of players who retired prior to the 1959 season in the NFL pension plan.

During his four years at Notre Dame, Hart caught 49 passes for 701 yards and 15 touchdowns, at that time a collegiate record.

[2][4] Hart was the captain of the 1949 Notre Dame team that compiled a perfect 10–0 record, outscored their opponents 360–86, and was recognized in the final AP Poll as the 1949 national champion.

[1] In February 1950, Hart married Lois Newyahr, his high school girlfriend, at St. Colman's Roman Catholic Church in Turtle Creek.