He began his professional music career playing trumpet, touring with, Mickey Fields, Lloyd Price, and substituting in the bands of Count Basie and Duke Ellington.
He worked with Jimmy Scott, LaVern Baker, Mickey & Sylvia, Clyde McPhatter, The Drifters, The Platters, and Fats Domino.
He moved to Hartford in the early 1960s to be close to New York City where he played in jazz clubs and worked as a studio musician.
In 1967, his desire to help quell the civil unrest of the mid 1960s contributed to the founding of the longest-running, outdoor, free jazz concert in the United States.
Brown worked with multiple jazz artists including George Benson, Max Roach, Bobby Hebb, Joe Henderson, Lou Donaldson, Grady Tate, Kenny Burrell, Philly Joe Jones, Walter Bishop Jr., Anita Bryant, Betty Carter, Junior Cook, Bill Cosby, during Cosby’s years with the Ebony Fashion Fair, Dexter Gordon, Al Haig, Bill Hardman, Barry Harris, Freddie Hubbard, Cab Calloway, Peter Nero, Larry Rivers, Hilton Ruiz, Nina Simone, Frank Sinatra, Mickey Tucker, Curtis Fuller, Gene Rodgers, Melba Liston, Dakota Staton, and Sarah Vaughan.