He also founded Dag Nasty in 1985,[3] was part of the original line-up of Samhain, and has had stints in Doggy Style, The Meatmen (with fellow Minor Threat member Lyle Preslar),[4] Government Issue,[5] and Junkyard (a hard rock band).

[8] Baker briefly toured with Me First and the Gimme Gimmes in 2005 and appeared on Canadian punk band Penelope's second album (Face au silence du monde).

[9] He has been a frequent guest guitarist on many songs and albums by artists as varied as Blood Bats, Tesco Vee, Ric Ocasek, Teenage Time Killers, Mind Over Four, Dangerous Toys, Pollen Art, Unwritten Law, Travis Cut, Lickity Split, Hot Water Music, Down By Law, Bash & Pop, Middle Aged Brigade, Careless, and many others.

The supergroup Foxhall Stacks, composed of Baker, Bill Barbot (Jawbox), Peter Moffett (Burning Airlines, Government Issue), and Jim Spellman (High Back Chairs, Velocity Girl) released their debut album, The Coming Collapse, in 2019.

[10] Another supergroup, Fake Names, formed with vocalist Dennis Lyxzén (Refused, The (International) Noise Conspiracy, INVSN), guitarist Michael Hampton (S.O.A., Embrace, One Last Wish), bassist Johnny Temple (Girls Against Boys, Soulside), and drummer Matt Schulz (Enon, Lab Partners, Holy Fuck), began touring in 2018[11] and released a self-titled debut album in 2020.