Born in Somerville, New Jersey, Mack attended the Juilliard School of Music, studying oboe with Harold Gomberg and Bruno Labate and then at the Curtis Institute of Music with Marcel Tabuteau, the longtime principal oboe of the Philadelphia Orchestra.

Mack was appointed by George Szell as principal oboist of the Cleveland Orchestra in 1965, succeeding Marc Lifschey, and remained there playing under Szell and his successors Lorin Maazel and Christoph von Dohnanyi until 2001 when he retired.

He also taught at the John Mack Oboe Camp, a yearly summer event established by Mack's student and former principal oboe of the New York Philharmonic, Joseph Robinson, in Little Switzerland, North Carolina.

[1] Mack began the John Mack Oboe Camp (JMOC) in 1976 to give more people access to excellent oboe teaching and mentoring; it is held each year in early summer at Wildacres in Little Switzerland, North Carolina, on the Blue Ridge Parkway.

The teaching legacy that Mack inspired and instilled is a summer tradition at Wildacres Retreat.