Following this early passion, Elvin joined his high school's black marching band, where he developed his foundation in rudiments.

By then, Jones was not entirely comfortable with Coltrane's new direction, especially as his polyrhythmic style clashed with the "multidirectional" approach of the group's second drummer, Rashied Ali.

[6] Notable among them was a trio formed with saxophonist and multi-instrumentalist Joe Farrell and (ex-Coltrane) bassist Jimmy Garrison, with whom he recorded the Blue Note albums Puttin' It Together and The Ultimate.

His final recording as a band leader, The Truth: Heard Live at the Blue Note, recorded in 1999 and issued in 2004, featured an enlarged version of his Jazz Machine—Antoine Roney (sax), Robin Eubanks (trombonist), Darren Barrett (trumpet), Carlos McKinney (piano), Gene Perla (bass), and guest saxophonist Michael Brecker.

[13] Among his last recordings was accompanying his brother, pianist Hank Jones, and bassist Richard Davis on an album titled Autumn Leaves under the name The Great Jazz Trio.

In 1969, Jones played drums for beat poet Allen Ginsberg's 1970 LP Songs of Innocence and Experience, a musical adaptation of William Blake's poetry collection of the same name.

[15] He appeared as the villain Job Cain in the 1971 musical Western film Zachariah,[16] in which he performed a drum solo after winning a saloon gunfight.

[19] Jones's sense of timing, polyrhythms, dynamics, timbre, and legato phrasing helped bring the drumset to the foreground.

In a 1970 profile published in Life Magazine, Albert Goldman dubbed Jones "the world's greatest rhythm drummer",[20] and his free-flowing style was a major influence on many leading drummers, including Christian Vander (Magma), Mitch Mitchell[21] (whom Jimi Hendrix called "my Elvin Jones"[22]), Ginger Baker,[23] Bill Bruford,[24] John Densmore (The Doors), Brian Viglione (Dresden Dolls and Violent Femmes) (for whom Elvin was his principal inspiration from age 11),[25] and Janet Weiss.