[1] Born in Sparks, Texas, Caldwell threw and batted right-handed, stood 6 feet 1 inch (1.85 m) tall and weighed 178 pounds (81 kg).

After entering pro ball in the Class D Texas Association as a 21-year-old in 1926 with the Temple Surgeons, Caldwell made his MLB debut with the Philadelphia Phillies on September 8, 1928, and pitched a 4–0, six-hit shutout over the Boston Braves, but lost four games after that, and was let go at the end of the season.

He reappeared successfully on September 22, 1935, outdueling Schoolboy Rowe in a three-hit, 1–0 shutout over the American League leading Detroit Tigers.

His fine season earned him another shot at the majors, where he won 13 games basically as reliever for the Chicago White Sox in 1945, at 40 years of age.

At the time of his retirement from baseball, he was pitching in the Big State League for the Corpus Christi Clippers when Mission Citrus Growers Inc. recalled him from a leave of absence to become the farm's general manager.