It was released in September 1984 as the lead single from their tenth and final studio album See Ya 'Round, and was a chart hit in New Zealand and Australia.

Finn's next group, Crowded House, re-recorded the song for the North American version of their self-titled debut album of 1986.

Finn sings both the Split Enz and Crowded House versions of "I Walk Away", and Paul Hester (who was a member of both groups) was the drummer on both.

Reviewing the Crowded House version, Junkee said, "There's something zeitgeisty about this one — it feels like U2 at their hungriest, propulsive and urgent in its delivery.

Hester puts on a showcase of tension and release, while Seymour holds down the low-end and Finn executes some truly underrated guitar work.