He entered the majors in 1939 with the St. Louis Browns, playing for them three years before joining the U.S. Navy Seabees during World War II.

He struggled with his control in his first four years (201 walks in 345.0 IP), but received a fifth chance in part to the World War II player shortage.

A week later, he extended the Browns season-opening winning streak to six games defeating the White Sox, 5–2, aiding his cause with a two-run home run.

Kramer finished with a 17–13 record and a 2.49 ERA, including a brilliant one-hitter shutout over the White Sox in September that gave St. Louis a half-game lead in front of the Yankees.

In November 1947, he was sent along Vern Stephens to the Boston Red Sox in exchange for Pete Layden, Joe Ostrowski, Roy Partee, Eddie Pellagrini, Al Widmar, Jim Wilson and cash.

In a 12-season career, Kramer posted a 95–103 record with 613 strikeouts and a 4.24 ERA in 322 appearances, including 215 starts, 88 complete games, 14 shutouts, seven saves and 16371⁄3 innings of work.