Formed in 1981, the band made their recording debut with the single "I'm Falling / Cold Life", released by Wax Trax!

[1] Signing a short-lived contract with Arista Records, the band released their debut studio album, With Sympathy, which hit the upper 90s in the Billboard 200, in May 1983.

Following a departure from Arista, Ministry returned on Wax Trax!, where several singles—including the club hit "(Every Day Is) Halloween"—were released in 1984–1985.

Both positively received by music press, these two albums hit the upper 160s in the Billboard 200, and were certified gold by RIAA.

During the tour in support of The Mind Is a Terrible Thing to Taste, the band recorded their debut live album, In Case You Didn't Feel Like Showing Up, released in 1990.