The Manor also included a small tract on the west side of the Hudson River opposite the promontory of Anthony's Nose, which van Cortlandt also purchased from the Indians.

[3] The Manor House [4] was built sometime before 1732 but was not any owner's principal residence until a grandson, Pierre Van Cortlandt, moved there in 1749.

He established a self-sustaining community[6] of an apple orchard, dairy farm, bee house, kiln, tavern, and carpenter and blacksmith shops.

Pierre sided with the colonies and the manor was used to assist the Continental Army, using its resources to make food and supplies.

Philip, becoming a brigadier general by the war's end, returned and, along with his sister, Catherine, brought the manor back to working order.