Built by Frederick Van Cortlandt and completed in 1749, the house is a 2+1⁄2-story Georgian building with a rubblestone facade and Georgian-style interiors.

The city government acquired the house in 1888 as part of the construction of Van Cortlandt Park and initially used the building as a police barracks.

[7] The house and surrounding landscape are preserved as part of Van Cortlandt Park, although the fields around the mansion date from the Parade Ground's construction in the late 19th century.

[11] The grounds surrounding the house were landscaped in what the historian Mary Lanman Ferris called "the Dutch manner of gardening".

[23][25] Following the takeover of New Netherland by the British in 1664,[26] the claim to the estate was awarded to van der Donck's brother-in-law, Elias Doughty, who proceeded to sell off the portions of the property.

[23][27] When Philipse's wife died, he remarried the daughter of Dutch brewer Oloff Stevense Van Cortlandt, herself a widow.

[31][32] The property's proximity to Tibbetts Brook, which drained into the Harlem River and Spuyten Duyvil Creek to the south, made it easy for Van Cortlandt to ship grain and timber products by water.

[58] Within the house, the family salted the pork and beef; cured the ham and bacon; and stored the various fruits that grew on the premises.

[35][56] The family often invited civilian and military officials to the mansion, serving lobsters from the Long Island Sound and hams from the estate's grounds.

[42] The Van Cortlandt family land served as a neutral ground during the American Revolutionary War[60] and was used by both the Loyalists and the Patriots.

[36][61] On May 30, 1775, the New York Provincial Congress placed James Van Cortlandt on a committee to create a report on whether it was feasible to build a fort near his family's house.

[31][43] Augustus Van Cortlandt hid city records under Vault Hill to protect them during the war, turning them over to the new American government afterward.

[43] After Washington's troops were defeated in the Battle of White Plains, British General William Howe made the house his headquarters on November 13, 1776,[69][70] placing it behind British-held ground.

[76] Records indicate that the family held events in the house as late as 1890, when Augustus Bibby Van Cortlandt married Ethyle Wilson there.

[93] The New York Herald Tribune described the house and surrounding property as having "for generations symbolized the vast wealth in real estate amassed by Oloff and Jacobus Van Cortlandt".

[95] The majority of the grain fields were converted into a sprawling lawn dubbed the "Parade Ground", while the Van Cortlandt House was preserved.

[98] Until 1896, the mansion also served as a barracks for the New York State Police, which had been assigned to guard the bison that roamed Van Cortlandt Park.

[103] In early 1896, the Society of Colonial Dames of the State of New York applied to the park commissioners for permission to repair the mansion and operate it as a historic house museum.

[47][58] A statue of National Guard major-general Josiah Porter was dedicated behind the house in 1902,[120] and the colonial garden adjacent to the mansion was completed in 1903.

[127] After a subsequent contract was approved, annex, consisting of a caretaker's apartment adjacent to the main house, was finished in 1916 or 1917, just before World War I.

[129] The architect Norman Isham was hired to renovate the mansion, which included restoring the fireplaces, adding paneling, moving the radiators, and installing interior shutters.

[135][136] During the late 1950s, a group led by New York State Assembly member Mildred F. Taylor found that the building was still in good condition.

[147][149] The work included a new 150-seat auditorium under the house; an expansion of the cellar for taller guests; new bathrooms; and mechanical, structural, and fire-safety upgrades.

[170] Despite the paucity of ornate ornamentation, one descendant, Catharine Van Cortlandt Mathews, wrote that the design "suggests to a large degree the substantial comfort of the era which it represents".

The eastern parlor had a cello, spinet, and piecrust table; the dining room had plates, set for a meal; and the kitchen had various utensils, as well as objects like a powder horn and a rifle.

[208] The museum hosted exhibits of colonial documents, paintings, and books in the 1920s,[209] and it showed glass, silverware, china, and pottery from the 17th and 18th centuries during the 1950s.

[210] The Colonial Dames has hosted live performances on the museum's behalf; for example, it staged a play at the Alvin Theatre in 1960 to raise money for the house.

[215] In the late 20th century, the house continued to present events such as concerts,[216] St. Nicholas Day carols,[217] children's programs, and historical lectures.

[15] A writer for The Christian Science Monitor wrote in 1915 that "this house helps us to picture their days of generous means and dignified living".

[225] Bronx historian Lloyd Ultan characterized the house in 1995 as being "highly significant to the history of the nation" due to its use during the American Revolutionary War.