During World War I he guested for Chesterfield, Crystal Palace and Huddersfield Town, although these appearances are not considered official.
In 1919 he turned professional and signed for Arsenal; according to club legend, Arsenal manager Leslie Knighton signed Baker after meeting him at the pit where he worked, thus beating rivals for the player's signature who were waiting at his home.
Baker played seventeen matches that season, but in 1920-21 he became a first-team regular; he also had trials for England, though it would be another seven years before he won a cap.
Baker finally won a major medal when he played in Arsenal's 1930 FA Cup final win over Huddersfield Town; by now he had nearly reached the end of his career.
His brother Jim Baker was also a footballer who played for Leeds United.