Williams was born in Galveston, Texas, and played tenor saxophone early in his life before picking up trumpet as a teenager.
After serving in the Air Force from 1952 to 1956, he toured Europe with Lionel Hampton, and upon his return took a master's degree at the Manhattan School of Music.
Among the musicians he worked with, apart from Mingus, are Oliver Nelson, Grant Green, Lou Donaldson, Yusef Lateef, Gigi Gryce, and Duke Jordan and the big bands of Duke Ellington, Gil Evans, Thad Jones and Mel Lewis, Sam Rivers and Clark Terry.
Williams also led bands under his own leadership, playing in New York jazz clubs such as Sweet Basil, the Village Vanguard, and Gerald's.
[1] With Ahmed Abdul-Malik With Mose Allison With Jaki Byard With Eddie "Lockjaw" Davis With Booker Ervin With Bill Evans and George Russell With Red Garland With Gigi Gryce With Slide Hampton With John Handy With Noah Howard With Sam Jones With Duke Jordan With Rahsaan Roland Kirk With Yusef Lateef With Les McCann With Jack McDuff With Charles McPherson With Carmen McRae With Charles Mingus With Mingus Dynasty With Oliver Nelson With John Patton With Jimmy Smith With The Thad Jones/Mel Lewis Orchestra With Charles Tolliver With Randy Weston With Leo Wright With Max Roach