Milner attended Southwest High School in Fort Worth, Texas, where he played baseball and football.

[1] He planned to attend Arizona State University,[1] but was selected in the seventh round of the 1978 MLB draft, held June 6–8,[2] by the Toronto Blue Jays.

[3] Milner joined the Blue Jays the next day, but did not make his major-league debut until June 23, in a road game against the Cleveland Indians.

[4] He became the first, and to date only, catcher to be brought directly to the major leagues after being signed since the MLB draft began in 1965, and the first Blue Jays player to do so.

[5] Making his debut at the age of 18 years, seven months, and six days, he remains the youngest person to play for the team.