It is housed in a handsome brick Classical Revival structure designed by George Swan and built in 1900–01, with a major expansion in 1985.

The original main block is a T-shaped structure with a hipped roof and a slightly projecting gable above an entry recessed under a round arch.

Windows are typically set in rectangular openings, with stone sills and keystoned lintels.

Construction of this building in 1900-01 was funded by Benjamin Smith as a memorial to his parents, with the proviso that the town acquire the land.

It was designed by George Swan, and is the second library in New Hampshire's Lakes Region that was built in the Classical Revival style.