He directed the 1998 film American History X. Kaye was born to an Haredi Jewish family in Stamford Hill, London, United Kingdom.

[1] He made his name as a director of television commercials with award-winning spots for British Rail InterCity ("Relax", 1988) and the Solid Fuel Advisory Council ("Furry Friends", 1988), as well as his 1993 advertisement for Dunlop Tyres ("Tested for the Unexpected") set to the sound of Venus in Furs by the Velvet Underground.

[3] Kaye made several music videos, including the video for "God's Gonna Cut You Down" by Johnny Cash, which won a Grammy Award, "Dani California" by Red Hot Chili Peppers, "What God Wants" by Roger Waters, and "Help Me" and "Runaway Train" by Soul Asylum.

Kaye spent $100,000 of his own money to take out 35 full-page ads in the Hollywood trade press denouncing Norton and the producer, using quotations from a variety of people from John Lennon to Abraham Lincoln.

He attended a meeting at New Line to which (to ease negotiations) he brought a Catholic priest, a Jewish rabbi and a Tibetan monk.

When the company offered him an additional eight weeks to re-cut the film, he said he had discovered a new vision and needed a year to remake it, and collaborated with Nobel Prize-winning poet Derek Walcott on new narration for the script.

Kaye's third feature film was a crime drama titled Black Water Transit starring Laurence Fishburne, Karl Urban, Evan Ross, Brittany Snow, and Stephen Dorff.