Williams played college football for the Arkansas Razorbacks and was selected in the fifth round of the 1952 NFL draft.

While attending Little Rock Central, he played high school football and was a two-time All-State selection.

He played defensive end and tackle with sporadic appearances at linebacker, a versatility that Chicago Tribune writer Cooper Rollow suggested led to his lack of accolades as he was never named All-Pro.

Nicknamed "Fat Freddy", he was popular among fans and media for his humor, frequently joking about various team-related situations; in 1959, when asked if the Bears' defensive line was the best in the league, he remarked, "I don't know if we're the best, but we sure are the ugliest.

[2] When the Redskins played Williams' former team that year, he was involved in a fight with Bears center Mike Pyle after Williams jumped offside and hit Pyle in the head with his forearm.