Headley commissioned the house and grounds from local architectural theorist and landscape designer Andrew Jackson Downing with assistance from his partner, English architect Calvert Vaux.

This concluded his property search between New York City and upstate, which he admired for its natural beauty, himself a native of Delaware County.

He made many interior changes on the upper floor, removing a dressing room and attic stairs to extend the central hallway.

[13] It joined the main house with the principal kitchen wing, allowing for more servant bed chambers and general space on the second floor.

Ownership shifted to John A. Corcoran in the early 20th century, who commissioned New York architecture firm Rogers & Haneman to construct the current back wing.

[15] The latter 20th century brought a period of historic home renovations to Newburgh, sparking a renewed interest in the abundance of local work by the partners.

[21] Of the house's initial appearance, Downing and Vaux wanted a spirited and irregular composition with simple details to appease Headley.

The heavily pierced bargeboard and tower cresting (now removed) are indicative of the architect's delicacy, as the rest of the exterior initially appeared rather plain.

[22] The foundation consists of Hudson bluestone, traditionally quarried in Kingston, but the remainder of the house is done in brick, over time, painted with several schemes.

[23][24] The architects despised plain white, and opted for a fawn color appropriate for the rural setting, translated by mixing ochre and sand shades, which mistakenly gave the impression of stucco.

"[28] It is through the tower's Moorish-style entranceway that one enters the house, with a repeated larger one into the main hallway, suggesting a Vaux implementation as seen at Olana.

An estimation of the landscape design's scale and beauty comes from the Headley's sale of the house, priced unusually high, indicating the grounds played a part in the value.

The deed of sale indicates multiples species of trees on the property, including chestnut, oak, cedar and cherry.

[34] An artificial pond of Downing's creation has since vanished, linking it with a still existing, albeit dry stream that forms the north boundary.

Willis's estate, the Vaux-designed Idlewild[36] in nearby Cornwall-on-Hudson, stood approximately 4 miles from Headley's Cedar Lawn.

"‘Round by Headley's’" we commonly call it – an upper road, along the bank of the Hudson, on which our friend the hero-grapher built his beautiful house, and 64 of the most charming of carriage-drives, avenued with cedars and country seats for miles.

As the finest rural outlet from the handsomest streets of Newburgh, we drove over it often, particularly with friends and strangers, whom we wished to impress agreeably with the scenery between Idlewild and there.