Baumann signed with the Red Sox in 1952 out of high school in his native St. Louis, receiving a $90,000 bonus from owner Tom Yawkey, who nicknamed him "Beau".

He won 10 of 11 decisions for the Triple-A Louisville Colonels in 1953, his second season in professional baseball, before being drafted into United States Army service during the Korean War.

In his debut, he earned a victory with 52⁄3 innings of scoreless relief, as Boston defeated the Detroit Tigers, 3–2.

He needed return trips to the minor leagues from 1956 to 1958 before making the Red Sox roster for the full 1959 campaign.

In 1960, as a member of the defending American League champions, he had a 13–6 mark for the White Sox, and led AL pitchers with a 2.67 ERA.