Gordon was born in Brooklyn, New York City, to a Jewish family, and later lived on Coney Island.

After attending public schools in New York City, Gordon worked in the Catskill Mountains at some of the resort hotels in the area.

After writing "Mister and Mississippi", Gordon decided he enjoyed puns on state names and later wrote "Delaware", which was a hit for Perry Como.

His 1956 hit for Patti Page, "Mama from the Train", was written to describe the love of a mother who had been born in the old country, but although the lyrics identify her as "Pennsylvania Dutch", the shifts into and out of a minor key mark the melody as Eastern European, and it was widely perceived as a tribute to a Yiddish-speaking mother.

"[2] Gordon told the Los Angeles Times that by 1960 the vogue for rhymed words and hummable melodies had passed, "So I became a tennis pro.