He was acquired by the Cardinals from the Topeka Jayhawks of the Western Association in 1927, and made his Major League debut less than two years later.

He finished 25th in voting for the 1931 National League MVP for playing in 131 games and having 447 at bats, 61 runs, 129 hits, 29 doubles, 5 triples, 1 home run, 62 RBI, 7 stolen bases, 54 walks, .289 batting average, .365 on-base percentage, .383 slugging percentage, 171 total bases and 4 sacrifice hits.

Gelbert's career was nearly ended when he severely injured his left ankle in a hunting accident, costing him two full seasons.

In nine seasons, Gelbert played in 876 games and had 2,869 at bats, 398 runs, 766 hits, 169 doubles, 43 triples, 17 home runs, 350 RBI, 34 stolen bases, 290 walks, .267 batting average, .336 on-base percentage, .374 slugging percentage, 1,072 total bases and 49 sacrifice hits.

After his playing career, Gelbert served as the coach of the Lafayette College baseball team for 21 years, from 1946 to 1966.