He was born in Goldsboro, North Carolina, and graduated from Tennessee State University, an historically black college in Nashville, where he played varsity baseball and basketball.

Once again making the All-Star team, Altman set personal bests in games played (147), hits (170), batting average (.318) and stolen bases (19); he also slugged 22 homers.

But the Cubs suffered through an embarrassing, ninth-place season, and in the 1962–1963 offseason, Altman became a major piece in a six-player trade with the St. Louis Cardinals that brought pitchers Larry Jackson and Lindy McDaniel to Chicago.

He became the 1963 Cardinals' starting right fielder and played a role in a pennant race that saw the Redbirds challenge the eventual world champion Los Angeles Dodgers into late September before a six-game losing streak doomed their chances.

But Altman's production declined, as he was platooned and started only against right-handed pitching; his average fell 44 points to .274, and he hit only nine home runs.

[6] In his nine-year major league career with the Cubs, Cardinals and Mets, Altman batted .269; his 832 hits included 132 doubles, 34 triples, and 101 home runs.