He joined the starting rotation in 1930, and posted a record of 39–37 from 1930 to 1932; on July 31, 1932, he pitched the first game ever at Cleveland Municipal Stadium.

In 1933 he finished second in the AL with an earned run average of 2.95, trailing teammate Monte Pearson (2.33); under modern rules (one inning pitched per games played by team), enacted in 1951, Harder would have been credited with the league lead, as Pearson only pitched 135 innings.

After developing a sore elbow and bursitis in his shoulder, he won at least 15 games each season from 1936 to 1939; he led AL pitchers in putouts in 1932, 1933, 1935 and 1938.

Harder was one of the most successful All-Stars of the 1930s, appearing in all four games from 1934 to 1937, and setting a record with 13 consecutive innings without an earned run.

From 1948 into the 1950s, he guided what became known as the Indians' "Big Four" pitching rotation, featuring Feller, Bob Lemon, Early Wynn and Mike Garcia; Harder had successfully transformed Lemon from an infielder into a top pitcher while working with him in the mid-1940s, and he taught future 300 game-winner Wynn the breaking ball and changeup.

Late in his tenure, prominent young pitchers as Sam McDowell, Tommy John and Luis Tiant would come under his guidance.

Harder is one of only four pitchers to pitch at least twenty seasons with one team (the others are Red Faber, Walter Johnson and Ted Lyons).