Born in Svir, Russian Empire[3] (now in Belarus) to a Jewish family, Gordon emigrated to the United States in 1908 at age five.

[4] Pursuing his parents' wish that he become a lawyer, he moved to New York in 1926 to attend Columbia Law School,[4][3] but began working in nightclubs and dropped out: "I learned to take my education where I could find it.

"[5] In 1932 Gordon opened his first venue, Village Fair, in the tradition of Viennese coffee houses as a place for artists and writers.

[4] The club hosted a who's who of jazz greats from the 1940s to the 1980s including John Coltrane, Sidney Bechet, Dinah Washington, Albert Ayler, Miles Davis, Wynton Marsalis, Henry Threadgill and Thelonious Monk (at the time an unknown, discovered by Gordon's wife Lorraine).

[2] In addition to the Village Vanguard, in 1943 Gordon opened the Blue Angel Supper Club in midtown Manhattan and was involved in its operation for fourteen years.