His mother Olivia Jones sang; his two older sisters studied piano; and his two younger brothers—Thad, a trumpeter, and Elvin, a drummer—also became prominent jazz musicians.

While playing with territory bands in Grand Rapids and Lansing in 1944, he met Lucky Thompson, who invited Jones to work in New York City at the Onyx Club with Hot Lips Page.

While practicing and studying the music he worked with John Kirby, Howard McGhee, Coleman Hawkins, Andy Kirk, and Billy Eckstine.

[10] In autumn 1947, he began touring in Norman Granz's Jazz at the Philharmonic package,[10] and from 1948 to 1953 he was accompanist for Ella Fitzgerald, and accompanying her in England in the fall of 1948,[11] developed a harmonic facility of extraordinary taste and sophistication.

Engagements with Artie Shaw and Benny Goodman followed, and recordings with artists such as Lester Young, Cannonball Adderley, and Wes Montgomery, in addition to being for a time, "house pianist" on the Savoy label.

During the late 1970s and the 1980s, Jones continued to record prolifically, as an unaccompanied soloist, in duos with other pianists (including John Lewis and Tommy Flanagan), and with various small ensembles, most notably the Great Jazz Trio.

In the early 1980s Jones held a residency as a solo pianist at the Cafe Ziegfeld and made a tour of Japan, where he performed and recorded with George Duvivier and Sonny Stitt.