The museum was founded by banker Matthias Hallenback Arnot (1833 - 1910), who bequeathed to the City of Elmira his art collection and the building that housed it.

[2] The original Greek-Revival mansion was built in 1833 by his father, John Arnot, Sr. Matthias built the picture gallery addition in the 1890s, which featured extensive gas lighting and a large skylight.

[citation needed] The European Art collection includes works by painters William-Adolphe Bouguereau, Jules Breton, Jan Brueghel the Elder, Gustave Courbet, Charles-François Daubigny, Anthonie de Lorme, Jean-Léon Gérôme, Claude Lorrain, and Jean-François Millet.

The American Art collection includes works by painters James Carroll Beckwith, George W Waters, Albert Bierstadt, Thomas Cole, Kenyon Cox, Jasper Francis Cropsey, Arthur B. Davies, Charles Warren Eaton, T. Alexander Harrison, William Stanley Haseltine, Robert Henri, George Inness, William Page, Ammi Phillips, Robert Street, Gilbert Stuart, Thomas Sully, Susan Waters, and minor Hudson River School artists.

Sculptors represented include Anna Hyatt Huntington, Chauncey Ives, Frederick MacMonnies, Larkin G. Mead, Joseph Mozier, and John H. Rogers.