[1][2] Williams was born in Geiger, Alabama and grew up in Lincoln, Nebraska.

He attended Lincoln High School where he participated in a variety of sports.

He set Nebraska state records in the 100 yard dash and the long jump, the latter of which was later broken by Gale Sayers.

He played running back on the Central State Bronchos football team.

[2] He also helped the Bronchos win the 1962 NAIA Football National Championship.