Smith was born to devout parents from Mobile, Alabama, and attended Wendell Phillips High School in Chicago, IL.
During the start of World War II, Smith briefly served in the Army, but was discharged for his height, which barely registered five feet.
Smith was known for his distinct baritone/tenor register, his unique delivery of songs which influenced the styles of other gospel singers such as Professor Alex Bradford and Rev.
Smith was often beckoned by music producers to release a solo album or to start his own group, but he refused and remained committed to The Roberta Martin Singers.
In the early 1940s, Smith composed the gospel blues song "I Know the Lord Will Make a Way, Oh Yes He Will", which is still popular among congregations today, and in 1949, became the business manager and booking agent for the Roberta Martin Singers.