Born in Chicago, Cleveland began singing as a boy soprano at Pilgrim Baptist Church where Thomas A. Dorsey was minister of music[3] and Roberta Martin was pianist for the choir.

He strained his vocal cords as a teenager while part of a local gospel group; this was said to contribute to the distinctive, gravelly voice which became a hallmark of his later in his career.

Soul Winners for Christ also included a young Richard D. Henton and was headquartered at Chicago's Greater Harvest Baptist Church; Louis H. Boddie was pastor.

Cleveland went to work with childhood friend Albertina Walker, founder of The Caravans, as a composer, arranger, pianist, and occasional singer as well as a narrator.

By 1960, Cleveland, who had incorporated blues riffs in his work – and what Anthony Heilbut described as "sheer funkiness" – had become associated with a new tenor in gospel music.

He became known by more than just the professionals within gospel music with his version of the Soul Stirrers' song, "The Love of God", backed by the Voices of Tabernacle from Detroit.

The live concert recording staple started with Reverend Lawrence Roberts, pastor of the First Baptist Church in Nutley, New Jersey, home of the famous Angelic Choir.

He returned to touring with the newly organized James Cleveland Singers, which included Odessa McCastle, Georgia White, Eugene Bryant, and Billy Preston, among others.

In 1965, Cleveland added Clyde Brown and Charles Barnett to his group, which by then was traveling extensively throughout the United States and abroad into the late 1960s, performing in all major venues.

After being nominated numerous times in previous years, James won his first Grammy Award with the Southern California Community Choir in 1975, for his 1974 album, In the Ghetto.

Additionally, Cleveland himself backed other acts, contributing to the recordings of well-known artists such as Aretha Franklin, Ray Charles, and Elton John.

[6] Initially unaware of the sound synchronization problem, Elliot mortgaged his own house to buy the film negative from Warner Brothers, but was able to use modern technology to bring audio and video together successfully.

[6] The completed documentary was finally released in April 2019, playing in select theaters across the world, with a premier at the New Temple Missionary Baptist Church itself.

[6] The historic documentary film Gospel, released in 1983, features James Cleveland for the first time on the motion picture screen, along with the Southern California Community Choir, Walter Hawkins & the Family, The Mighty Clouds of Joy, Shirley Caesar, and The Clark Sisters.

Excerpts from the film soundtrack was released on Savoy Records and became a top seller, with Cleveland having the majority feature on side two of the album.

Throughout his recording career, Cleveland started a trend and was famously known for taking some of the greatest pop, R&B, and soul songs and transforming them into gospel anthems.

Another memorable gospel rendition was of Barry Manilow's "I Write the Songs," which he recorded on his 1977 Grammy-winning album, Live at Carnegie Hall.

[13] Following Cleveland's death in February 1991, a number of controversies arose surrounding his estate, which at the time was estimated to be worth between $4 and $6 million.