Within a short time, he began to work at the Jazz Showcase in Chicago, where he appeared with drummer Wilbur Campbell and pianist Willie Pickens.
Throughout the years he has worked with George Coleman, Lee Konitz, Bobby Hutcherson, McCoy Tyner, Sonny Fortune, Ira Sullivan, Junior Mance, David "Fathead" Newman, Charles McPherson, Branford Marsalis, Benny Golson, Jackie McLean, Sonny Stitt, Eddie "Lockjaw" Davis, Al Cohn, Eddie Harris and Les McCann.
In addition, he has collaborated with guitarists Kenny Burrell, Joe Pass, and Tal Farlow, as well as the drummers Ed Thigpen and Jack DeJohnette, and trumpeters Donald Byrd and Harry "Sweets" Edison, among others.
His album Larry Gray 1,2,3 ..., featuring his guitar trio's performances of his original compositions, on Chicago Sessions Records, was released in December 2008.
He has recorded with Curtis Fuller, Ira Sullivan, Lin Halliday, Jodie Christian, Willie Pickens, Nicholas Payton, Randy Brecker, Bunky Green, Bob Moses, Laurence Nugent, John Williams and Bohola, Linda Eder, Dennis DeYoung, Jim Peterik, Peter Cetera, and Michael Smith.