He received Alabama All-State and Class A "Back of the Year" honors in 1956 when he served as the team's captain.

[1] Elmore attended and played college football at the University of Mississippi, during which the Rebels posted a 29-3-1 record.

[1] In 1961, Elmore helped lead Ole Miss to a 9-2 record that season and was an All-America first-team selection.

[1] Elmore also played baseball for the Rebels as a center fielder and helped Ole Miss win 1960 Southeastern Conference championship.

[1][2] Elmore was an assistant coach for one year at Southeastern Louisiana University,[2] then was the vice president of Boyles Moak Brickell Marchetti Insurance, Inc.[1] Elmore had a wife, Linda; a son, Jimmy; a daughter, Sharon Duncan; and four grandchildren.