Its first director, elected in 1883, was artist Carl Ludwig Brandt, who spent winters in Savannah.

Its entrance is a form typical of architect William Jay, with a projecting four-column portico that is accessed via side-facing stairs.

The site on which it was built previously housed the official residence of Georgia's colonial royal governors.

In 1875 Alexander's sister, Mary, bequeathed the house, including its furnishings and family collections, to the Georgia Historical Society, which opened the first art museum in the southeastern United States here in 1886.

In front of the building are statues of Rembrandt, Rubens, Phidias, Raphael, and Michelangelo (see the link to Commons below).