[4] The academy maintained a museum that displayed animals, fossils, minerals, and plants, as well as ethnological artifacts such as weapons, costume, tools, statuary, and musical instruments.

As built, it was a two-story structure, faced in granite on the front (Essex Street facade), and brick on the sides, measuring about 45 by 100 feet (14 m × 30 m).

[14] The interior of the building has been repeatedly altered over the years, losing a significant amount of original detail in the process.

The first floor, which was originally occupied by retail businesses, was adapted as a museum space in 1867–69, following its acquisition by the Peabody Academy of Science.

The upper level of the hall, designed as a ballroom and auditorium space, has always been retained as a large open gallery, but lost a significant amount of historic fabric in the 1860s alterations.