Baker led the NFL in scoring in 1957, was a first team All-Pro in 1966, and was selected four times to participate in the league's postseason Pro Bowl All-Star game.

[7][8] He helped his team win the 1948 Oregon state championship in basketball[3] and also lettered in baseball.

[9][10] In 1952, Baker participated in the East-West Shrine Game in San Francisco, setting a record for the longest punt (72 yards).

[13] In the 1950 Civil War game, Baker had a 59-yard touchdown run on a fake punt, and ran for over 100 yards.

[14][15] In 1953, he played sparingly in his first season with the Washington Redskins, under coach Curly Lambeau, before spending two years out of professional football, while serving his military service at Fort Ord.

[11][16] In 1956, although he was initially being considered for the right halfback position, he was asked to become the team's kicker after Vic Janowicz suffered a serious brain injury in an automobile accident that ended his athletic career.

[18] He was given the nickname "Sugarfoot",[19] after leading the NFL in field goals (17),[20] starting an 11-year streak of averaging at least 40 yards per punt attempt[4] and being named to the Pro Bowl.

[4] On April 25, 1960, he was traded to the Cleveland Browns in exchange for Fran O'Brien and Robert Khayat.

[27] In 1960, he relinquished his fullback duties with the Cleveland Browns and would replace the retired Groza, who left after the 1959 season because of a back injury.

[citation needed] Baker played two seasons as a punter and kicker for the Dallas Cowboys,[4] until his disregard for the team rules and discipline wore thin with head coach Tom Landry.

[40][41][4] On March, 20, 1964, he was traded to the Philadelphia Eagles along with John Meyers and Lynn Hoyem, in exchange for wide receiver Tommy McDonald.

[47] Upon retiring he was the number two scorer in NFL history (977 points)[46] and held the record of scoring in 110 straight games.

[9] Baker met his wife Sarah at Corvalis High School, and both attended OSU.