Built in 1873 for a congregation founded in the mid-18th century, it is a well-preserved design of James Renwick Jr. and a good example of late 19th-century Gothic Revival architecture.

The church is a Gothic Revival structure, built out of granite quarried at Greenwich, Connecticut with lighter-colored trim from Ohio.

At the southeast corner stands a square tower, with stone buttresses at its lower level, a belfry at the second stage, and a pyramidal cap that has rounded turrets at the corners.

The present building is its fourth sanctuary, the congregation having outgrown each of the first three to be built.

The church altar is adorned by a reredos created by sculptor Gutzon Borglum and installed in 1913.