It was originally named Carrollcliffe and was built as the residence of "General" Howard Carroll, a journalist, playwright and businessman, with intention that it should be reflect Norman castles in Wales, Scotland and Ireland.

New owners Hanspeter and Steffi Walder of Tarrytown bought the property along with a group of investors in 1992.

Their goal, Hanspeter says, is the operation of a hotel and restaurant that can be a haven for the stressed executive or professional desiring to relax in a setting that is warm, inviting, and majestic.

[6] In 2003, Elite Hotels, a limited liability company formed by C. Dean Metropoulos, bought the property, then called The Castle at Tarrytown for $10.9 million, according to a newspaper article at the time, which described it as "a 31-room inn on 10.1 acres".

Overlooking the Hudson River valley, it is located on the highest point of Westchester County, about 20 miles (32 km) north of the George Washington Bridge.