It is the seventh track and the lead single from her third studio album, To Bring You My Love, released in February 1995 on Island Records.

The song diverged significantly from Harvey's former indie rock and punk blues influence, opting to use electronic instruments such as a synthesized organ and an orchestral ensemble.

The lyrics narrate the story of a woman drowning her daughter and reference the traditional American folk song "Salty Dog Blues".

The musicians in the ensemble include Sonia Slany on violin, Jocelyn Pook and Jules Singleton on viola, and Sian Bell on cello.

The refrain was based on Lead Belly's rendition of the traditional American folk song "Salty Dog Blues".

[5] The song charted in two charts in the United States; peaking at number 48 on Billboard' Hot 100 Airplay,[6] and reaching number 2 on the Modern Rock Tracks[7] "Down by the Water" became a modern rock radio hit due to extensive rotation on MTV and college radio stations.

"[8] Robert Hilburn of the Los Angeles Times noted the song in 1995 as one of To Bring You My Love's "most gripping moments", adding "she speaks with the captivating clarity and force of someone reaching for a final, life-saving anchor".

[10] In 2005, Diplo sampled "Down by the Water" for a remix of the Mike Jones single "Still Tippin",[11] and then again the following year on Plastic Little's indie rap track "Now I Hollar".