Kelly was born on a farm near Fonda, Iowa on July 3, 1893; he was the third of five children in the family.

[1] In high school, he was a hurdler and middle-distance runner and played football and basketball.

[1] He studied at Buena Vista College, Morningside College, and the University of South Dakota; after graduating from South Dakota, he returned to Fonda as an all-sports high school coach, and then to Buena Vista as a collegiate basketball, football and track coach.

[1] His teams at Buena Vista put up good records in all three sports, including winning the 1925–26 Iowa Conference basketball championship; he left in 1926, moving to DePaul University.

[1][5][6] At Minnesota, Kelly introduced an improved discus throw technique based on speed and good footwork; two of his pupils, Bob Fitch and Fortune Gordien, set new world records in the discus with the "Minnesota Whip".