Palmer grew up in Breckenridge, Texas and played football, baseball and was on the track and field team at Albany High School near his hometown.

[5] Palmer, by then a junior and the defensive captain, played well in the game, but TCU lost 40–26 to a Georgia team led by quarterback Frank Sinkwich.

[1][5] The Chicago Bears of the National Football League (NFL) selected Palmer in the 1943 draft, but he decided to put off his professional career to serve in the U.S. Marines during World War II.

[8][9] With the Yankees, Palmer was part of a potent offense that featured Sinkwich and star halfback Spec Sanders, who in 1947 had 3,631 running, passing and kick return yards and 19 touchdowns.

[1] The team finished with a 6–8 record, and Palmer was traded after the season to the Browns in exchange for the draft rights to center Alex Sarkisian.

[7] After Rymkus retired in 1951, Palmer saw increased playing time on offense as he shared tackle duties with John Sandusky.

[2] He also served on the Cleburne city council, was president of the local Lion's Club and worked for a variety of other civic groups and projects.

[2] His son Scott played on a University of Texas football team that won the national championship in 1969, and had a two-year career as a tackle in the NFL.