John made his Arsenal first-team debut on 28 October 1922 in a 2–1 defeat at home to Newcastle United and quickly became a regular, succeeding Tom Whittaker at left half.
John was displaced from the Arsenal side in 1923–24 thanks to competition from Billy Blyth and Andrew Young, but after being switched to left back to cover for Andy Kennedy, he was a near ever-present in 1924–25.
By this time he was one of the senior members of the Arsenal squad, and mentored many of the club's younger new arrivals, such as Alex James.
Although John played for Arsenal until his retirement in 1938, for the final three years of his career he was mainly a reserve player, having lost his place to Wilf Copping, and thus missed out on a medal in the Gunners' League win of 1934–35.
He had an unsuccessful spell as manager at Plainmoor, winning only 7 of the 28 games he was in charge for and left his post in November 1950.