After serving in World War II, he started working in radio, appearing on stations in Missouri, Arkansas, Michigan, and Kentucky.
Wheeler signed as a solo artist to Sun Records in 1957 and went on tour with headliner Hank Snow, and Faron Young, the Wilburn Brothers, Mother Maybelle & the Carter Sisters, Slim Whitman, Jerry Lee Lewis, Johnny Cash, Elvis Presley, and Carl Perkins.
His biggest success was the 1973 hit "John's Been Shucking My Corn", though the achievements of his daughter Karen Wheeler would eclipse his own in the decade.
He played on OKeh records, and was a Studio Musician for such groups as Gospel duo, Slim & Zella Mae Cox.He bought a guitar shop in Nashville in the late 1970s, and played at the Grand Ole Opry, as a Member of Roy Acuff's Smoky Mountain Boys band, from 1962-until his death.
Jimmie Snow's Grand Ole Gospel Radio Show in 1984 when he collapsed and died of a heart attack while standing in the Circle.