A prolific session musician, McCoy performed on many recordings by established artists, including Elvis Presley (on eight of his film sound tracks), Bob Dylan, Johnny Cash, Chet Atkins, Waylon Jennings, Roy Orbison, Leon Russell, and Loretta Lynn.

In the recording industry, he was known as the "utility man" because of his ability to play with sufficient skill on many different instruments in addition to the harmonica; for example, he played trumpet on Dylan's "Rainy Day Women ♯12 & 35", saxophone on Roy Orbison's "Oh, Pretty Woman", and bass harmonica on Simon and Garfunkel's "The Boxer".

On Elvis Presley's 1965 soundtrack album Harum Scarum, he played guitar, harmonica, organ, and vibraphone.

[2] For 19 years, McCoy worked as music director for Nashville's popular television show, Hee Haw.

[3] His memoir, Fifty Cents and a Box Top– The Creative Life of Nashville Session Musician Charlie McCoy, was published in 2017.

In 1949, at age 8, McCoy saw an advertisement in a comic book, that read: "Learn to play harmonica in seven days or your money back.

[5]: 5  As he got older, he devoted himself to learning blues harmonica by listening to late-night radio broadcasts originating out of Nashville's WLAC.

While attending Southwest Miami high school, he put together a rock and roll band called "Charlie McCoy and the Agendas" as a guitarist and singer.

[6] At age sixteen he accompanied a friend to a Miami-based country music barn dance radio show called the "Old South Jamboree".

[7] McCoy, who still wanted to make a career in music, decided to return to Nashville, a difficult decision, because it meant quitting college, and would disappoint his father.

He then received a call from the booking agent Jim Denney, who informed him that Archie Bleyer of Cadence Records had listened to McCoy's tapes and wanted to sign him.

Next, McCoy joined Wayne Moss as a bass player, performing at Fort Campbell in Kentucky.

[8] Chet Atkins heard one of McCoy's demo tapes and immediately hired him as a staff musician in May, 1961.

Finally, he played a motif quietly for veteran session guitarist, Harold Bradley, who said, "Hey guys, Charlie has a great idea.

[4] He said, “After the song hit the radio, my phone started ringing ... and thank God, 61-years later, it's still ringing.”[1] He continued to record for the Monument label without a written contract.

[8] Tex Davis, the promoter for Monument Records, was persuaded by Charlie Dillard of WPFA to release "Today I Started Loving You Again" from McCoy's second LP as a single.

[5]: 80  The success of this impromptu session is credited in part with Dylan later agreeing to come to Nashville to record, and doing so against the wishes of his label and management executives.

"[1] Dylan returned to Nashville to Columbia Studios in late 1967 to record John Wesley Harding, with McCoy playing bass.

[12] McCoy went on to record with many other artists including Elvis Presley, Perry Como, Gordon Lightfoot, Joan Baez, Steve Miller Band, Johnny Cash, Roy Clark, Buffy Sainte-Marie, Kris Kristofferson, Paul Simon, Barefoot Jerry, on Ringo Starr's Beaucoups of Blues, on Gene Summers' Gene Summers in Nashville and 12 Golden Country Greats by Ween.

In 2017, The West Virginia University Press published Fifty Cents and a Box Top: The Creative Life of Nashville Session Musician Charlie McCoy.

[5] During the Saturday night broadcast on June 11, 2022, McCoy was invited to become a member of the Grand Ole Opry by Vince Gill.

His second granddaughter did the artwork for three album covers (Somewhere Over The Rainbow, Smooth Sailing, Celtic Dreams) and sang on one of his Christmas CDs.