In the 1960s, Smith accompanied Aretha Franklin, Nina Simone, Lloyd Price, and Nat King Cole; he worked with Sam Rivers from 1964–1976 and with Gil Evans again from 1968 to 1976.

Through the 1970s, Smith played with Andrew White, Julius Hemphill, Muhal Richard Abrams, Nancy Wilson, Quincy Jones, Count Basie, and Carmen McRae.

Other credits include extensive work with rock and pop musicians and time spent with Anthony Braxton, Charles Mingus, Henry Threadgill, Van Morrison, and Joe Zawinul.

Smith taught in the New York City public school system from 1958 to 1968, at Third Street Settlement from 1960 to 1967, at Adelphi University in 1970–1971, and at SUNY-Old Westbury from 1971.

With Gene Ammons and Sonny Stitt With Sam Most, Kenny Barron, George Mraz With Kenny Barron With Anthony Braxton With Rob Brown With Jaki Byard With Composer's Workshop Ensemble With Gil Evans With Art Farmer With Benny Golson With Billy Harper With Julius Hemphill With J. J. Johnson With Hubert Laws With Giuseppi Logan With Herbie Mann With Ken McIntyre With Van Morrison With M'Boom With Jack McDuff With Charles Mingus With Buddy Montgomery With Jimmy Owens With Jerome Richardson With Sam Rivers With Alan Silva With Jimmy Smith With Charles Tolliver With Phil Upchurch With David S. Ware With The Tony Williams Lifetime With Odean Pope With Joseph Daley With Blue Reality Quartet (Joe McPhee/ Michael Marcus/Jay Rosen)