His family lived on the east side of the city and he eventually attended Flint Central High School.
He then attended Michigan State University (MSU) attaining a bachelor's degree in police administration with honors in 1960.
AutoWorld encouraged other downtown development such as the Hyatt Regency Hotel, which opened in 1981; Water Street Pavilion; and Windmill Place.
With his top aides, he formed the Rutherford Group, Inc. which went on to run Mrs D's Sandwich Shop and a coffee service both for delivery to Total gas stations, Sunshine convenience and Action Auto stores throughout southeast Michigan, the three Harpos pizzeria locations, and a downtown Coney Island cart.
[2] After attending a Toughman Contest in 1984 at the IMA Sports Arena, Rutherford fought a robber for several minutes but was robbed of "a substantial amount of money" and received minor injuries.
[2] Running for Genesee County Clerk as a Democrat in 1984, he lost in the primary to John H. Trecha, the incumbent, by 73 votes, 13,444 to 13,371.
Rutherford ran once again for mayor of Flint in 1987, but came in third out of seven candidates in the primary with Sharp and Matthew S. Collier continuing on to the general election.
[2] Moving back to the private sector in 1989, he worked as head of public relations for Windmill Place, then owned by businessman Don Williamson.
[8] In 2003, Rutherford became the director of the Flint Downtown Development Authority and continued in this capacity under mayor Don Williamson[9] serving until 2008.