It was built in 1868 for William Skinner, the owner of a successful silk spinning and textile business, and is named for the abundant wisteria vines which cascade across its eastern facade.

The lot is ringed by a low stone retaining wall, which is topped by iron fencing, with entrances on the Beech Street side providing vehicular access to the carriage house and main drive.

[9] The oldest part of the mansion was built while Skinner lived in the nearby town of Williamsburg, and was moved to Holyoke in 1874 when he relocated his business there after a devastating flood.

[11][12] The building originally was crowned with a large cupola and spire, however by the end of the 1960s this was removed, several windows were filled, and by 1980 the entrance was altered to its present appearance today.

Although a new cupola would remain absent, following $1 million in refurbishments provided by the United Bank Foundation, Massachusetts Cultural Council, and the city, the building was restored largely within its original architectural style and reopened as the archival research center in October 2009.

[17][18] The initial mission of the Association was "to awaken and maintain an active interest in the Practical Sciences, and to aid generally in connection with Arts, Agricultural, Manufacturers and Commerce", and its bylaws would introduce the museum, which was to be curated by the city's librarian.

When the library opened in 1902, the museum found its first home in the east wing on the second floor, and by that time had acquired the "Sherman Indian Collection" of indigenous stone implements from one of its members, William J Howes.

Schurr, a naturalist and taxidermist, would assemble a comprehensive collection of specimens from the Pioneer Valley's flora, fauna, minerals, and fossils, and served in this post until his death in 1951.

With Belle Skinner's untimely death in 1928, her brothers, Joseph and William, would work to preserve and share the treasures of her estate, including her world-renowned rare instrument collection.

[29][30] For the three decades that the music room was open, Wistariahurst would host many social clubs, colleges, and musicians, including Serge Koussevitzky, director and conductor of the Boston Symphony Orchestra, who described it as "a collection of superlatives".

[33] Among other notable pieces in this collection was a spinet built by Pascal Taskin for Marie Antoinette, a virginal made for Nell Gwyn, one of the only surviving examples of keyboards from Carolean England, and a Yuan dynasty era guqin bearing the signature of noted Chinese poet and musician Zhao Mengfu.

Though the effort to capture the qualities of these historic instruments was praised by music critics, the release was also panned by some, including one reviewer in Gramophone who lamented “unfortunately the recording techniques used are so poor that the net result is an often unacceptable sound”.

[40] The group and its director, Mary Preiss, would successfully get the Wistariahurst estate on the National Register of Historic Places in 1973,[41] and during this period its collections of natural history specimens, art, and artifacts were prominently featured in the halls of the house, as well as a number of notable exhibitions, including an example of Claude Monet's Houses of Parliament, loaned from the Metropolitan Museum of Art in 1968.

Shortly before his resignation, Mayor William Taupier demanded the private Library Corporation relinquish control of the Wistariahurst estate to the city's Historical Commission, in return for public funding.

This collection, once housed in Wistariahurst in the mid-20th century held British, Irish, and American Impressionist paintings, including examples by Holyoke local William Chadwick, William Merritt Chase, Frederic Edwin Church, F. O. C. Darley, Francis Day, Frank Duveneck, John Joseph Enneking, Edward Gay, Mauritz de Haas, Howard Russell Butler, Joseph Goodhue Chandler, Eastman Johnson, Régis François Gignoux, Charles P. Gruppé, Hugh Bolton Jones, Alfred Jacob Miller, Edward Moran, John George Brown, N. C. Wyeth, Maitland Armstrong, Ivan Olinsky, Thomas Gainsborough, Arthur Fitzwilliam Tait, John Henry Twachtman.

The museum features original leather wall coverings, columns, elaborate woodwork and an interesting tale of how two generations perceived and used the house very differently.

[57][58] In addition to seasonal exhibits, gallery shows, and programming with partner institutions, the museum also maintains numerous items found in its permanent collections.