The song was written by the band's eponymous vocalist along with original guitarist and bassist Daisy Berkowitz and Gidget Gein, respectively, and was produced by Manson with Trent Reznor.

The song features saxophone played by Sugarsmack vocalist Hope Nicholls, and an audio sample of the televised suicide of R. Budd Dwyer.

"Get Your Gunn" was blamed by the political right in the United States for the Columbine High School massacre, and multiple critics have deemed it one of the most controversial songs of all time.

At his trial, Griffin claimed to have been manipulated into killing Gunn by John Burt, a pro-life leader and former member of the Ku Klux Klan.

In 1999, Manson wrote in Rolling Stone that Gunn's death "was the ultimate hypocrisy I witnessed growing up: that these people killed someone in the name of being 'prolife.

"[4] Commenting on both "Get Your Gunn" and "Lunchbox", Manson noted that "The somewhat positive messages of these songs are usually the ones that sensationalists misinterpret as promoting the very things I am decrying.

[14] Alec Chillingworth of Metal Hammer found the song's "wacky, stomping hook" reminiscent of the music of Jack Off Jill.

[5] The track features Manson singing "Pseudo-morals work real well, On the talk shows for the weak, But your selective judgements, And goodguy badges, Don't mean a fuck to me".

[12] Jim Louvau of the Phoenix New Times named it one of his favorite Marilyn Manson songs alongside "Long Hard Road Out of Hell" (1997).

[4] Bloody Disgusting's Brad Miska opined that "In retrospect, its level of cheese is of legend, but in 1994 Portrait was some dark and weird shit (see 'Cake and Sodomy', 'My Monkey', 'Get Your Gun' [sic], etc.).

[16] Jon Wiederhorn of Loudwire praised the track for "driving the record" and helping the band to retain the loyal fan-base that it had cultivated prior to the release of Portrait of an American Family.

[18] Sandra Schulman of the Sun-Sentinel said that the song is "Horrifyingly good, loopy music, with discordant riffs that crawl under your skin and stay there.

"[7] Dazed's Daisy Jones described "Get Your Gunn", Nirvana's "Rape Me" (1993), The Prodigy's "Smack My Bitch Up" (1997) and Lady Gaga and R. Kelly's "Do What U Want" (2013) as "tracks that provoked, thrilled and shocked their listeners.

"[9] Conversely, The Chicago Maroon's Matt Zakosek called the track "weary Goth-rock junk" and criticized its inclusion on the greatest hits album Lest We Forget: The Best Of (2004).

Jones wrote that the song created an amount of controversy comparable to that sparked by "Fuck tha Police" (1988) by N.W.A and "Blurred Lines" (2013) by Robin Thicke.

[7] Dan Epstein of Revolver wrote that "Marilyn Manson might be best known as a musician, but he's also made his mark as a video artist, simultaneously captivating and disgusting viewers since 1994's 'Get Your Gunn' clip all the way up to this year's nympho-nuns-with-guns visual 'We Know Where You Fucking Live'.