James Caldwell, a Presbyterian minister and active supporter of the Patriot cause during the American Revolutionary War.

An original parsonage dating to 1730 was burned in 1780 by a Loyalist mob,[4] and later that year Caldwell's wife Hannah was killed by British soldiers in the house during the Battle of Connecticut Farms.

In 1782, with the war over, what is now Connecticut Farms Presbyterian Church built the present building.

It continued to serve as a home for its pastors until the 20th century, when the church built one closer to the building.

Several state grants have been made for renovations and upkeep of the aging structure.