[11][12] Consisting of members of the Jon Spencer Blues Explosion, the Didjits, and Tar, Deadly Cupcake was formed by Albini.

[15][16] The Knoxville News Sentinel stated: "Schneider still uses his distinctive style—an emphatic shout/sing delivery with an ironic tone—but now it's accompanied by heavy angst as he sings about endangered freedon ('Helicopter'), betrayal ('Lick', 'Secret Sharer') and destruction ('Bulldozer').

"[20] Rolling Stone concluded that "Albini manages to make all these guitars sound authentic and contemporary, but Just Fred is as much a nod to the late-70s underground scene that spawned the B-52's, when the punk rock of the Ramones seemed just fine alongside the quirky pop of bands like Blondie and Talking Heads.

"[2] The Atlanta Journal-Constitution determined that "a few tracks illuminate surprising similarities between Schneider's arch yelp and Johnny Rotten's—'Secret Sharer' could be a Never Mind the Bollocks outtake.

"[23] AllMusic called the album "a surprisingly enjoyable fusion of Schneider's bizarre charm and lean, loud alternative rock that proudly illustrates his punk roots.