[2] From its formal classical nucleus to the naturalized wilderness of its forest, literally every square foot of the estate was shaped by the ideas of Colonel Clarence S. Wadsworth and his architects.

After marrying Katharine Fearing Hubbard, Col. Wadsworth began amassing land in the western part of Middletown that would eventually become his 600-acre (2.4 km2) estate.

Starting in 1900, thousands of trees and shrubs were planted to change pastures and orchards into a naturalistic woodland setting around the mansion.

The mansion's architect was Francis Hoppin, who was classically trained at Brown University, M.I.T., and the Ecole des Beaux-Arts in Paris.

The Wadsworth family was in residence at Long Hill in the spring and fall, but maintained homes in New York, Palm Beach, Bar Harbor, Chicago, and Bermuda.

The Wadsworth family maintained Long Hill until 1947, when it was sold to Our Lady of the Cenacle, a Roman Catholic religious institute who used it as a retreat center for 40 years.

The bank that repossessed the estate did not adequately secure the building and it was not long before vandals, satanic worshippers, and homeless people frequented the house.