He was a member of the group between the single "Life Begins at the Hop" (1979) and early sessions for the album Apple Venus Volume 1 (1999), contributing guitar, keyboards, and occasional string arrangements.

He joined XTC as guitarist immediately prior to the recording of the Drums and Wires LP in 1979, when he replaced Barry Andrews, eventually leaving the band in 1999.

Since leaving XTC Gregory has been much in demand as a session musician with a number of artists, including Peter Gabriel, Aimee Mann, Cud, Marc Almond, Bingo Durango, Johnny Hates Jazz, Jason Donovan, Martin Newell, Louis Philippe, Lulu, Mark Owen, R. Stevie Moore and others.

Gregory, who has been regularly involved in Steve Hogarth's h-Band, has also contributed to works by Porcupine Tree, including string arrangements on their sixth album, Lightbulb Sun, and for Dublin group Pugwash.

Gregory was also a member of the group Tin Spirits, which featured ex Stamford Amp singer Mark Kilminster and also guitarist Daniel Steinhardt and drummer Douglas Mussard.