It is a small wooden building in the Picturesque mode of the Gothic Revival architectural style dating to the mid-19th century.

It was originally built not on its present site but in what is now midtown Manhattan, for a congregation that had split off from another church downtown.

The church property is a 12.5-acre (5.1 ha) parcel on the west side of an unpaved driveway off the south side of Bingham Road in the southern section of the Town of Marlborough, roughly a mile and a half (3 km) southwest of the town's major settlement, the unincorporated hamlet of Marlboro.

In this portion a sand-surfaced volleyball court has been built along the driveway, with some playground equipment present in the warmer months.

The driveway bends south through a parking area to the southeast of the church to a small square half-court with basketball hoop on the west.

The church itself is a cruciform wood frame one-and-a-half–story building on a foundation of concrete on the front and sides and mortared stone in the rear.

The downward slope of the ground exposes the basement on the sides and rear, giving the effect of an additional story.

Gently curved wooden pews with scroll armrests and recessed end panels flank the center aisle, leading to the altar.

Inside the entrance to the south room is an intricately carved wooden surround with religious motifs that supports the organ pipes.

[1] In the early 1830s, some members of the Oliver Street Baptist Church (now the Mariner's Temple) in Lower Manhattan split from that congregation.

On that land they built the original 1834 Amity Street Baptist Church, a temple-style Greek Revival building with six Ionic columns along its front facade.

Samuel Dunbar, the architect, is believed to have emulated his earlier design for the Thirteenth Street Presbyterian Church.

[1] Lower Manhattan's growth and transformation in the early industrial era was rapidly making it less desirable for churches, and many sold their original properties in that area, using the proceeds to buy and build further uptown.

Its Picturesque styling, enhanced even further by its original board-and-batten siding, was then a common mode for many American Protestant churches.

Richard Upjohn, an Episcopalian immigrant from England, popularized the Gothic style for larger churches of his denomination.

Baptists found more modestly scaled, restrained versions worked for them stylistically, and many variations were built.

He bought the 200-acre (81 ha)[4] Marlborough property for retreats from the city in 1893, holding annual meetings of the Brotherhood of the Kingdom,[5] a group he had helped found along with Social Gospel proponent Walter Rauschenbusch, there starting that year, and coming there more often with his wife Nellie after their 1896 marriage.

The chapel remained, and in 1905 Williams had it disassembled and shipped to Ulster County, where it was rebuilt at the current location.