He excelled as a fielder, becoming the National League's top third baseman in the late 1910s and early 1920s, and captained championship teams with the 1919 Reds and 1922 Giants.

He set major league records for career fielding average (.967) and double plays (278), and upon retiring ranked third in NL history in games (1,299) and assists (2,554) and fourth in putouts (1,456) and total chances (4,146) at third base.

At 5 feet 8 inches and 158 pounds, he appeared younger than his 23 years; in his first major league at bat, umpire Bill Klem questioned whether McGraw had mistakenly sent a batboy to the plate, but Groh came through with a base hit.

He also led the league in hits (182) and on-base percentage (.385), and was second in runs (91), walks (71) and total bases (246) and sixth in slugging average (.411).

1918 was an even better season in various ways, as despite a season curtailed by World War I and the influenza epidemic, he tied Billy Nash's 1890 major league record of 37 double plays, also leading the league in putouts (180) and fielding average (.969); Pie Traynor would set a new record with 41 double plays for the 1925 Pittsburgh Pirates.

He had begun the year with a long contract holdout, only joining the team in June for $10,000 (less than the $12,000 he had requested) on the condition that he would be immediately traded to the Giants.

In December, after the Giants had won the World Series, he was finally sent to New York for two players and cash (varying reports give amounts from $100,000 to $250,000).

The Giants won the pennant again, becoming the first team to remain in first place all season, but lost the World Series in six games to the Yankees, with Groh batting only .182.

Groh, who had suffered a knee injury late in the season, made just a single pinch-hitting appearance, singling in the 11th inning of the final game; the Giants lost 4–3 in the 12th when, with one out and runners on first and second, Earl McNeely hit a ground ball to rookie third baseman Freddie Lindstrom which apparently struck a pebble and bounced over his head for a double.

His final major league appearance was as a pinch-hitter in the ninth inning of Game 3 of the 1927 World Series against the Yankees, in which the Pirates were swept; he popped up to pitcher Herb Pennock.

He later worked as a racetrack cashier, and was among the baseball figures interviewed for Lawrence Ritter's landmark 1966 book, The Glory of Their Times.