After graduating from high school, he worked as a clerk in a mail order business before launching one of the most successful and prolific careers from Tin Pan Alley.

In 1913 he began a productive partnership with the well-established composer Egbert Van Alstyne, with whom he created several notable hits of the era, including "Memories" and, along with Tony Jackson, "Pretty Baby."

[2] Throughout the 1920s, Kahn continued to contribute to Broadway scores such as Holka Polka (1925), Kitty's Kisses (1926), Artists and Models (1927), Whoopee!

[2] He also collaborated with co-lyricist Ira Gershwin and with some of the finest composers, including Grace LeBoy Kahn (his wife), Richard A. Whiting, Buddy DeSylva, Al Jolson, Raymond Egan, Ted Fio Rito, Ernie Erdman, Neil Moret, Vincent Youmans, George Gershwin, Harry Akst, Harry M. Woods, Edward Eliscu, Victor Schertzinger, Arthur Johnston, Bronisław Kaper, Jerome Kern, Walter Jurmann, Sigmund Romberg, and Harry Warren, though his primary collaborator was Walter Donaldson.

His catalog contained some of the greatest collections of songs from the first half of the 20th century, and it is for this reason that he was inducted into the Songwriters Hall of Fame in 1970, nearly 30 years after his death.