Built for Francis Cottenet, a wealthy New York merchant, it was later owned and renovated by Cyrus West Field, John Jacob Astor III and Manhattan College.

It was originally located on 65 acres (26 ha), the sole house between the Albany Post Road (now US 9) and the river in what has since become the Ardsley-on-Hudson section of Irvington.

[1] After Cottenet's death in 1884, it passed to another wealthy local, Cyrus West Field, who had to sell a year later due to some unexpected financial setbacks.

During his brief ownership, he placed in the deed restrictions on the property's future use, banning it from being used for certain purposes, such as industry, education, or sale to "disreputable" individuals.

[1] His heirs sold the house to Manhattan College, which spent the next few years trying to clear Field's restrictions from the deed and convert the home to institutional use.

Failing, the college sold to Amzi Lorenzo Barber, the "Asphalt King", a pioneer in paving as well as other businesses that would become prominent in the coming century.

[1] Since then, Nuits has been the home of Robert Hewitt, Jr., engineer John Wiley, Gillies Coffee Company owner James Henry Schmelzel,[1] and investment banker Martin W.

[1] The arched main entrance is located in the middle of the south elevation, a three-bay facade with central tower and balustraded balcony with hooded window above.