Designed by architect H. Neill Wilson with landscaping by Frederick Law Olmsted, the mansion and farm buildings were built for Anson Phelps Stokes in 1893.

Today the historic district primarily encompasses Berkshire Country Day School, which acquired its campus from the Stokes family in 1963.

Nathaniel Hawthorne originated the name "Shadow Brook" in reference to a small stream that lies to the west and south of the mansion site.

The new structure, completed in 1893 at a cost of $500,000, was shaped like the letter 'L' with its point aimed south and comprised two dominant styles involving English Tudor timbering and stone.

The wife of a Vanderbilt family member leased the property briefly in 1916 following her husband's death, and the mansion was sold in 1916 by Shotter's debtors to Andrew Carnegie for $300,000.

"[3] The replacement building, 400 yards (370 m) west of Shadowbrook's ruins, was completed and in use by 1958 "with a nondescript brick and concrete exterior and much painted cinderblock inside" according to Shea.