It was originally established in 1955 in the Normaltown neighborhood of Athens, but was later located at the corner of Hawthorne and Oglethorpe Avenues.

Slaughter hired Danny Self to run the business and eventually sold it to Self while keeping the real estate.

Allen's got its "World Famous" nickname because of its proximity to the Navy Supply School.

In 2007, two University of Georgia alumni, Mark Hammond and Hilt Moree, III, acquired the rights to Allen's from Slaughter and re-opened it in a new location with much of the original memorabilia, recipes, and live music.

Notable patrons of Allen's include the late writer Lewis Grizzard, professional wrestler Bill Goldberg, lead guitarist/singer John Bell of Athens band Widespread Panic, R.E.M.