[1] Andy was born in Athens, Alabama, where his father Guy was chief pitmaster for Limestone County.

He was a four-sport athlete at Athens High School playing football, basketball, track and baseball, serving as co-captain of the 1951 team and winning letters for each of his four years he completed.

Nelson spent two years as a player and defensive coach with the Harrisburg Capitols of the Atlantic Coast Football League, the Colts' farm team, where he led the league in pass defense before going on to be named the head football coach.

Nelson was also defensive coordinator for Philadelphia Bell in the World Football League from 1974–75.

[citation needed] Andy Nelson's Barbecue is a BBQ restaurant in Cockeysville, Maryland.