[1] First published in 1834, it endures in modern hymnals to a setting written by John Goss in 1868, and remains one of the most popular hymns in English-speaking denominations.

[2] The text of the hymn was first published in Lyte's The Spirit of the Psalms (1834),[2] a publication intended for the use of his own congregation in southern England.

[3] It remains extremely popular and John Richard Watson notes that "it is hard to find a major hymnbook that does not include it".

While, in the mid-nineteenth century, hymn writers usually kept their metrical settings of psalm texts as close as possible to the original, Lyte instead decided to maintain the spirit of the words while freely paraphrasing them.

The result speaks, in an imaginative fashion, with "beautiful imagery and thoughtful prose", of themes such as the Love of God, healing and forgiveness,[1] including the repeated exclamations "Praise Him!

[2] This was an instant success, a report in the 1869 Musical Times stating that "it is at once the most beautiful and dignified hymn tune which has lately come under our notice".

[2][b] An alternative tune is "Regent Square", originally written by Henry Smart for "Glory be to God the Father" by Horatius Bonar.