The Only Ones came out of London during the first wave of punk (1976–77) and, rather like the New York bands the Heartbreakers and Television with whom they later toured, suffered from being too musical for lumpen-punk but too "new" for conservative record business sensibilities.

[2] Perry's guitar style is noted for a combination of attack and melody, a mixture that the UK music magazine Sounds described as being "very superb".

He has also worked with Johnny Thunders, Marianne Faithfull, Grateful Dead lyricist Robert Hunter, Alejandro Escovedo, the Sisters of Mercy, Mick Green (of The Pirates), Robert Palmer, Evan Dando of The Lemonheads, Martin Stephenson and the Daintees, Wayne Kramer, Screaming Lord Sutch, James Williamson, Nick Kent & The Subterraneans, Michael Nyman, JC Carroll, The Members, Freddie Stevenson, Heidi Berry, and Jayne County.

The band's first major label single, "Another Girl, Another Planet," appeared on CBS to immediate and almost universal critical acclaim but crawled no higher than the lower reaches of the Top 60, achieving only #56 UK.

Working with producer Colin Thurston, the band later achieved a top forty album with Baby's Got a Gun (#37 UK).

In late February 2007, the Only Ones reformed,[2] appearing at All Tomorrow's Parties and a sold-out show at the Shepherd's Bush Empire.

Perry, who played lead guitar on the Lemonheads' album Varshons, joined Evan Dando onstage at the Ben & Jerry's Summer Sundae festival at Clapham Common, London on 27 July 2008.

[10] More recently, Perry has played live and recorded with Martin Stephenson and the Daintees, The Members frontman JC Carroll, Mark Keds' East London band Deadcuts, and Texan singer Alejandro Escovedo.

He contributed an article on the influential acoustic guitarist Bert Jansch to a collection of 33.3 authors published in 2019 by Bloomsbury.