Eugene Kelly formed The Vaselines in Glasgow, Scotland in 1986 with Frances McKee and was a member until 1989 when the band disbanded the week their first and only full-length album, Dum-Dum, was released.

In 1990, Kelly formed Captain America but was forced to change the name after Marvel Comics, who owned the trademark of the superhero name, threatened them with legal action.

[3] In 2006, he toured with Isobel Campbell, replacing Mark Lanegan when performing songs from Ballad of the Broken Seas and supporting Mogwai and The Lemonheads in Europe.

In the summer of 2006, Eugene Kelly and Frances McKee took to the stage to perform a set of The Vaselines songs, as part of a joint tour to promote their individual solo albums.

This led to The Vaselines reforming (minus the old rhythm section) on 24 April 2008 for a charity show for the Malawi Orphan Support group at Glasgow's MONO venue.