[1][2] Alto saxophonist and vocalist Steve Sostak and guitarist Chris Daly met while attending the University of Notre Dame, where they played together in Check Engine between 1993 and 1994.

[3] The two relocated to Chicago after graduating, where they recruited drummer Scott Anna and bass guitarist Matt Alicea, forming Sweep the Leg Johnny.

[3][11] Allmusic described their sound as a combination of "dissonant rock, progressive music, jazz, noise and punk", while the Chicago Tribune described the band as "art-punk", stating that the band "combines the complexity of art rock, the sledgehammer forcefulness of punk, brooding melodies and countless hours of toil into one of the city's most distinctive sounds", describing their music as "a roller-coaster ride of jarring guitar riffs and blaring free jazz saxophone, abruptly changing time signatures and dramatic, dynamic shifts".

[3][5] The Stranger described them as "the only band in America that can make a saxophone sound like it belongs in the middle of intimidating post-punk guitars and apocalyptic vocals".

[12] The Riverfront Times described the band's music as "a jazz-laden style of indie rock that featured plenty of mathy twists and turns".

[13] Reviewing Tomorrow We Will Run Faster, Allmusic writer Rick Anderson viewed their music as falling "somewhere between the dry, academic precision of 1980s King Crimson and the sprawling art-punk clamor of Mission of Burma".