Born in Loris, South Carolina (near the Myrtle Beach area), Brooks grew up on a tobacco farm.

A 1960 dairy science graduate of Clemson University, Brooks spent time in the United States Army before founding Naturally Fresh Foods, a condiment and salad dressing manufacturer in the Atlanta area, in 1967.

Under Brooks' leadership, Hooters expanded from half a dozen restaurants (in and around Clearwater, Florida) in the mid-1980s; to over 430 stores worldwide (including Taiwan, Venezuela, and Switzerland).

The Robert H. Brooks Performing Arts Center at Clemson, completed during the 1993-94 school year, was named in his honor following a large donation.

His eldest son, Mark, from his first marriage predeceased him when he was killed on April 1, 1993 in a plane crash with 1992 NASCAR Winston Cup Series champion, Alan Kulwicki.