Written by Nick Cave and Mick Harvey, the song was released as a single on 31 July 1981 through 4AD record label, with the B-side "Blast Off".

[3] The track and its B-side were recorded on the same night with a session at the Townhouse Studios with producer Nick Launay, who was commissioned by 4AD because of his work on Public Image Ltd's Flowers of Romance (1981).

The term goth did not exist at that time, certainly not in the way we would use it these days, but I will say that recording a song called 'Release The Bats' with people who looked like vampires was pretty fucking exciting!

During the rest of the recording, the band members regularly disappeared to the bathroom, which according to Launay "added to the fuel and edginess of the night," and notable arguments broke out, mainly aiming at drummer Phill Calvert.

"[3] AllMusic's Amy Hanson categorized the track as a "deep rolling bass-led cacophony,"[3] while Sasha Frere-Jones of New Yorker argued that its "rhythm section vamps around a menacing bass figure while the guitar lurches back and forth, unsure of whether to make noise or form chords.