It is a 1-1/2 story roughly rectangular masonry structure, built out of a combination of local fieldstone, red sandstone, and yellow brick.

At the front right corner is a projecting rounded area, capped by a bell-shaped roof.

The interior retains a number of period features, but others have been lost due to the building's conversion to non-library use.

Northrop was an eclectic architect conversant in all of the popular styles of the time, and he clearly based this design on the landmark Crane Memorial Library of H. H. Richardson.

Construction of the building was funded by Henry Augustus Taylor, a railroad executive who moved to Milford in 1890.