Six years later, it had incorporated and hired Richard Upjohn, a prominent architect noted for his churches, to design a building.

The cornerstone—located at the southwest buttress of the tower—was laid on June 14, 1859 and the church was consecrated by Bishop Horatio Potter on October 25, 1860.

Holy Comforter is a Gothic-Revival structure with walls of local Ulster County bluestone from across the river and trim of New Jersey brownstone.

[1][3] An addition to the north was built in 1867 with funds provided by William Augustus Davies in memory of his late wife, Sarah Van Wagenen.

It is also a contributing property to the Mill Street-North Clover Street Historic District listed on the Register later that year.