It occupies a lot covering an entire city block on the southeast corner of Piedmont Avenue and Ponce de Leon Avenue in Midtown Atlanta, just north of the SoNo neighborhood.

The house incorporates references to the Peters family, such as tiles depicting the Philadelphia Fish and Chowder Society founded by Edward's great-grandfather Richard Peters, Jr. Edward's father, also named Richard Peters, was instrumental in the founding and transformation of the small town of Marthasvillle into Atlanta.

After the death of Peters' daughter-in-law in 1970 the house was threatened with demolition, but it became The Mansion restaurant, operated by Bill Swearingen three years later.

[3] The interiors have been largely preserved, although a fire in 2000 caused the restaurant to close.

A full restoration of Ivy Hall was undertaken, reopening on October 3, 2008.