He attended high school in Rockford, Illinois, and spent two years at Berklee College of Music in Boston.
Bach's work with the band began on tours for Mermaid Avenue; the first album he played on was Summerteeth.
He contributed extensively to the albums Yankee Hotel Foxtrot and A Ghost Is Born, playing keyboards, guitar, bass, and saxophone.
[5] Bach has played with Joan of Arc, Theaster Gates, Will Oldham, Rob Mazurek, The Autumn Defense, Liz Phair, Beth Orton, Andrew Bird, Iron and Wine, and Dwain Story.
He maintained an 18-month residency at the Hideout, in Chicago, with singing partner Edward Burch, and co-produced Sonny Smith's 2006 album Fruitvale.
In 2009, Bach and Gates set to music the poems of slave potter Dave Drake, which were performed at the Milwaukee Art Museum.