Dr. Carter, a New York City pediatrician, built the school on his farm with the intention of providing a country environment and an educational program for his physically handicapped son, Herbert Swift Carter, Jr.

He named the school for Sir William Harvey (1578–1657), personal physician of King Charles I who is considered one of the fathers of modern medical science.

[3] John L. Miner was appointed as the school's first headmaster when its doors opened in October 1916, with an enrollment of twelve boys.

In 1959, the school lost its Hawthorne campus due to construction of a cloverleaf highway interchange on the Taconic and Sprain Parkways, and moved to its current 100-acre (40 ha) campus on the former Sylvan Weil Estate in Katonah.

The fourth and fifth grades were eliminated in 1983 and 1984, and girls were admitted to the middle school beginning in 1984.

[2] During his tenure, in March 2005 a new arts center was completed[5] to provide an environment for studio, music and dramatic-arts education, as well as a flexible performance theater.

The Student Council meets regularly to discuss improvements in school life and to plan all-school activities such as Family Weekend and holiday parties.