After leaving the Labor Party in the late 1980s, Kaye worked for independent community candidates and developed a passion for "sensible urban planning, genuine community consultation and participatory democracy" and he joined the Greens Party in 1997.

[2] From 1998 to 2001, Kaye was the Greens policy coordinator and, from 1999 to 2002, was policy advisor to Greens MLC Lee Rhiannon, leading campaigns for public education; sustainable transport; the urban, rural, and natural environments; workers' rights; and developer donations to political parties.

[1] In the 2004 federal election Kaye was the Greens lead candidate for the Australian Senate from New South Wales.

As lead candidate, the Greens vote increased to 7.3% but, due to less-favourable preference flows, he failed to gain a seat by a margin of 0.5%.

Kaye's portfolio responsibilities included Premier & Cabinet, Treasury, Finance, Education and Training, Energy, Health Services, Science & Medical Research, Water Utilities, Fair Trading, Gaming and Racing, Infrastructure, and Commerce.