He played college football for the Notre Dame Fighting Irish before transferring to the Ohio State Buckeyes.
After college, Graham had a lengthy career in the NFL with the New York Giants in two separate stints, as well as starting for the Arizona Cardinals and the Pittsburgh Steelers.
However, the option-run offense implemented by coach Lou Holtz was an imperfect fit for the drop-back passer, and after his sophomore year he transferred to Ohio State, where he was the starter for the 1991 campaign in which the Buckeyes finished 8–4.
Graham returned to the Giants in 1998 to serve as backup to Danny Kanell, who had led the team to the NFC East title the previous year.
Graham led the Giants to two wins in his first three starts, then secured a major upset over the then-undefeated and eventual Super Bowl champion Denver Broncos in Week 15 by throwing a late touchdown pass to Amani Toomer to clinch the victory.
Graham was released during the off-season (February 10, 2000) due to his lack of consistency and to create room under the salary cap.
In February 2000 Graham was signed by the Pittsburgh Steelers on a $5.1 million deal over three years to replace Mike Tomczak, who had been released in the off-season, as the backup for Kordell Stewart.