Now based in New York City,[1] the group currently consists of singer Kassie Carlson, drummer Peter Negroponte, guitarist Arian Shafiee, keyboardist Jake Lichter, and bassist Zach Lewellyn.

[4] In 2011, Andy Allen and Peter Negroponte formed an early instrumental version of Guerilla Toss that played a mix of free jazz and punk.

[5][6] Guerilla Toss toured extensively, nationally, and within Boston's DIY community by performing at many show houses such as Gay Gardens, where Carlson and Negroponte were both residents on separate occasions.

[9] The group then released an EP later that year—Kicked Back into the Crypt, a split LP with noise rock band The Sediment Club that was co-released through Sophomore Lounge Records and Feeding Tube Records[10] Guerilla Toss began to incorporate disco and funk elements into their sound as shown with the release of their debut LP Gay Disco through NNA Tapes in December 2013.

[15] Guerilla Toss wrapped up the year by going on a US / Europe tour that included the fourth-to-last show at New York City warehouse venue Death By Audio with Thee Oh Sees.

[22][23][24] Around this time, the band began varying their setlists and jamming during their live performances in addition to encouraging tapers to come and check out their shows, citing the Grateful Dead as an inspiration.

[28] Popular YouTuber and music critic Anthony Fantano gave the album an 8/10, saying "Thanks to some clearer production, more up-front performances, and a zany new wave influence, GT ULTRA is the most I've ever enjoyed a Guerilla Toss album"[29] and included it on his "Loved List: 2017"[30] On July 18, 2018, the band released a music video for the song "Meteorological", alongside an announcement on their official Facebook page.

[42][43] On October 28, 2022, Guerilla Toss released the single "Heathen Money" on a compilation called "The Eleventh Hour: Songs For Climate Justice", curated by Sub Pop and film director Adam McKay.

[48] On February 20, 2024, it was announced that Guerilla Toss would be opening for Primus and Coheed and Cambria on their summer co-headline tour, [49] with Katzman rejoining the band on second guitar and vocals.

Throughout their DFA years, critics noted the addition of psych,[53] krautrock, new wave,[54] funk and dance music elements to the bands sound, often being compared to artists like Talking Heads,[28] The B-52's,[55] Boredoms,[56] Brian Eno, and Grace Jones.

[60][61] They have cited artists such as Charli XCX,[62] Andy Stott,[39] Caroline Polachek,[63] Arca,[39] Arthur Russell, and Boards Of Canada[64] as recent influences.

[66] Carlson's older brother is singer Jonah Jenkins of Only Living Witness[67] and Milligram,[68] as well as several other Boston based underground punk and hardcore bands throughout the 90's.

[64] Carlson's lyrics often deal with subject matter such as substance abuse and recovery,[76][77] growing up in poverty,[78] religious fanaticism,[79] the psychedelic experience,[80] mental illness,[81] technology, climate change,[82] self motivation,[83] depression[84] and alien abduction.