The cover art for the single is a painting by Malcolm Bucknall called "Old Indian and White Poodle".

[5][6] The British magazine Music Week described "Oh, the Guilt" as a "rougher, underproduced example of their pop-metal" and "Puss" as "a fiercer, messier hardcore thang".

[9] David Yow, vocalist of the Jesus Lizard, later said of the release: "We had to figure out, well, do we want to do this and look like we're riding on Nirvana's coattails, or we could just do it and not worry about it, which is what we ended up doing.

"[1] Nirvana recorded an instrumental demo version of "Oh, the Guilt" on January 1, 1991, at The Music Source studios in Seattle, Washington, but it remains officially unreleased.

All of these also remain officially unreleased except for "Aneurysm" and "Even In His Youth" which featured as B-sides to the "Smells Like Teen Spirit" single, and "All Apologies" which was released on the 20th anniversary deluxe and super-deluxe versions of the In Utero album in 2013.