[2] Kelsey attended Cwm School but left at a young age to work alongside his father and qualified as a crane driver.
[2] He was spotted by an ex-Arsenal player, Les Morris, whilst Kelsey was playing for his local side, Winch Wen, then in the Swansea & District League.
Kelsey made a total of four appearances that season, but was dropped once Swindin returned and did not feature in Arsenal's 1951–52 run to the FA Cup final.
[4] With a rugged build and consistently solid catching, Kelsey managed to fully oust Swindin from the side (Swindin making only two appearances in 1953–54) and went on to be Arsenal's first-choice goalkeeper for the next eight seasons; only a broken arm sustained in an FA Cup tie against Sheffield United in 1959 put Kelsey out of the Arsenal side for any considerable amount of time, with Jim Standen taking over in the meantime.
Although his playing career coincided for the most part with a trophyless run for Arsenal – their best finish being third in 1958–59 – and thus he did not win further honours with them, he is still regarded by the club as one of their greatest-ever goalkeepers.