The museum currently contains a collection of over 4,500 American and European paintings, sculptures, and works on paper, housed in three buildings: the 1818 Telfair Academy (formerly the Telfair family home); the 1816 Owens-Thomas House & Slave Quarters, which are both National Historic Landmarks designed by British architect William Jay in the early nineteenth century; and the contemporary Jepson Center for the Arts, designed by Moshe Safdie and completed in 2006.

Each of the museum's three buildings houses a collection corresponding to the era in which it was built.

The Telfair Academy contains two nineteenth-century period rooms, and it houses nineteenth- and twentieth-century American and European art from the museum's permanent collection including paintings, works on paper, sculpture, and decorative arts.

The Owens-Thomas House & Slave Quarters contains a decorative arts collection composed primarily of Owens family furnishings, along with American and European objects dating from 1750 to 1830.

Additionally the site includes intact urban slave quarters and a parterre garden.