He was a scene painter at the Old Vic Theatre in Waterloo and illustrated regularly for the NME, i-D, Literary Review, Time Out and International Musician magazines between 1987 and 1989.

[citation needed] Kaye formed and sang in many bands, including the dark psychedelic outfit We Are Pleb, who played extensively in Camden during 1988–89 (at the same time as Blur and Suede) and had a penchant for setting the stage on fire.

Kaye was signed to Go Discs in 1992 with a group called TV Eye (formed with ex-members of the band Eat), which released two singles, "Killer Fly" and "Eradicator".

[citation needed] In 1993, Kaye filmed a prototype Dennis Pennis, interviewing his own band on a late-night indie music show on Granada TV called Transmission.

He had an office in White Hart Lane and designed in-house merchandise for Spurs, Derby County, Southampton and Aston Villa for Danish sportswear brand Hummel International (doing caricatures of Paul Gascoigne for school lunchboxes etc.).

"[citation needed] In 1994, Kaye convinced his old friend Anthony Hines (a car mechanic and some time roadie for TV Eye) to help him write Dennis Pennis when he was offered the job on The Sunday Show.

[5] (Hines was later poached by Sacha Baron Cohen to write for Ali G on The Eleven O'Clock Show and went on to receive an Oscar nomination for co-writing Borat in 2006).

[2] With dyed red hair, gaudy jackets adorned with punk-style badges, and thick glasses, Pennis stood out from the crowd and asked celebrities atypical questions, ranging from playful to cruel.

[5] Originally, the celebrities would be mainly British stars harassed at assorted London-based events, such as actor Hugh Grant, TV host Ulrika Jonsson, and sports pundit Des Lynam.

[7][8] His victims from this point on were more renowned, the most famous of whom were Arnold Schwarzenegger, Demi Moore, Kevin Costner, Morgan Freeman and Bruce Willis, amid a raft of other Hollywood A-list stars.

"[10] A special one hour video-only feature was released in 1997 called Dennis Pennis RIP: Too Rude to Live, which saw the character killed off, after Kaye decided the rewards were not worth the effort.

He also briefly presented a BBC2 quiz show, Liar, in which six contestants would all have a supposed claim to fame and the studio audience voted on which one they believed was telling the truth.

[14] His role as deaf DJ Frankie Wilde in the 2005 mockumentary It's All Gone Pete Tong[15] won him the Film Discovery Jury Award at the 2005 US Comedy Arts Festival.

[25] Kaye's other television credits include The Trial of Elizabeth Gadge, an episode of Reece Shearsmith and Steve Pemberton's anthology series Inside No.

In 2019, Kaye debuted as hospital chaplain Daniel Booth in the ITV dramedy Cold Feet, and played a psychiatrist in the Netflix comedy series After Life.

[1] In January 2009, Kaye wrote an article for The Guardian in which he called for peace between Israel and the Palestinians after his mother-in-law was killed by a Hamas rocket strike on the Gvar'am kibbutz.