Hunt was commissioned by his brother-in-law, Joseph Howland, to design a home for a subscription library he donated to the city.

The completed building has much in common with the Stick style summer homes in Newport, Rhode Island, that Hunt designed at this early stage of his career.

The blocks to the west and north are urban and densely developed with larger mixed-use buildings, on the south side of the street.

The Madam Brett Homestead, a 1709 stone house also listed on the Register, is near the west end of the block.

Another listed building, Beacon's post office, is a block and a half west along Main Street.

The building itself is a two-story three-by-five-bay brick structure on a foundation of bluestone and granite quarried at nearby Breakneck Ridge.

Atop is a six-gabled roof covered in Delaware slate pierced by a brick chimney on the west side.

Floors are of English cane felt overlaid with hemlock, to dampen sound, and topped with strips of Georgia pine.

[1] Joseph Howland, a Civil War general and former New York State Treasurer whose Tioronda estate was a mile (1.6 km) south of the building's location in what was then the village of Matteawan, commissioned the building from his brother-in-law, Richard Morris Hunt, late in 1871.

The woodwork around the main entrance also shows hints of the emerging Stick style; there was one more at the gables, but it has been lost.

Seven years later, more space was needed, and the upper gallery was added, the last significant change to the building.

[1] Even after the village of Matteawan, where it was located, merged with nearby Fishkill Landing at the beginning of the 20th century to become today's city of Beacon, the library remained subscribers-only.

The library continued to serve the city and its residents from the building until the 1970s, when the growth of the former outpaced the capacity of the latter.

[6] In 2007, the center installed geothermal heating to cut its energy costs and reduce the use of fossil fuels.

A system using water in vertical pipes to be dug 250 feet (76 m) into the earth beneath the center will both heat and cool the building.