Architect Alexander Parris designed the museum building, which is built of Quincy granite and opened in 1824.

Russell Warren constructed a wooden portico in 1834, which had Doric columns supporting a triangular pediment.

In 1922 the original wooden portico was replaced by the present six-column Greek Revival temple front, which was designed by McKim Mead & White.

[2] In 2008, an addition was added to the museum along with a new sign, activities, and advertising throughout the downtown area.

The museum owns the remnants of the Sparrow Hawk, the only known remains of a trans-Atlantic 17th-century ship, which wrecked off of Cape Cod in 1626.