Lyrically, Morris compliments the band for "[expressing] their anarchist views […] in a way that is honest and open," as well as for including "silly stories" in songs such as "The Exorcist," "Me and the Boys," and "Randy, I Don't Need You."

Morris concludes his review by stating, "Randy seems to have made a very successful album that will most likely be overlooked by many, but it's their loss, as all the pieces truly are in place."

He points out the band's drastic "reinvention" compared to their previous style, which was more in line with skate punk outfits like Satanic Surfers.

On the contrary, Dubar points out that You Can't Keep a Good Band Down is "noticeably less aggressive, much more catchy, and that's not a bad thing at all."

He compliments Randy's "straightforward" lyrical style and avoidance of metaphors and poetry in favor of "tackling realistic ideas and offering real-world solutions."