It is located at 53 South Cliff, in an architecturally distinguished Richardsonian Romanesque building designed by Connecticut architect George Keller and completed in 1892 as a gift from Caroline Phelps Stokes.

[1] The Ansonia Library stands in a mainly residential area a short way east of downtown Ansonia, on a triangular parcel at the junction of South Cliff Street and Cottage Avenue.

Its asymmetrical plan is visually dominated by a square tower on the left side, with a clock in its upper stage and a pyramidal copper roof.

The main entrance is set near the center of the South Cliff facade, recessed under a large round arch.

[2] The building was donated by Caroline Phelps Stokes as a memorial to her parents and to her maternal grandfather Anson Greene Phelps, a locally prominent industrialist who founded the Ansonia Brass Works, and for whom the city is named.