Born in Montreal, Quebec,[1] he studied piano 1945–49 at the Conservatoire de musique du Québec, where Arthur Letondal was his teacher.
[2] During those years Kaye also performed with his own groups and was a dominant participant on the Montreal jazz scene, as a pianist and also as a valve trombonist or trumpeter in the bands of Steve Garrick, Gilbert "Buck" Lacombe, and others.
Kaye was also highly in demand by the great vocalists of the day; he performed as accompanist with the likes of Tony Bennett, Pearl Bailey, Edith Piaf, Sammy Davis Jr., Josephine Baker, and Mel Tormé.
In the late 1970s he worked frequently in a trio format with bassist Jean Cyr and various drummers, occasionally adding trumpeter Charles Ellison, or vocalist Barbara Reney to complete a quartet.
Kaye influenced many of the jazz performers of his day,[4] and also left a legacy of unrecorded compositions which are today held in the Concordia University Archives.