[1] Built to overlook an important road on the western edge of the North York Moors, the castle fell into ruin as early as the mid-14th century.

[2] At the time of the Domesday Book, Whorlton was recorded as belonging to Robert, Count of Mortain, the half-brother of William the Conqueror.

Darcy carried out substantial changes to the castle and levelled the motte to provide a base for a new keep with a fortified gatehouse, built a short distance to the east.

[11] At some point in the late 16th or early 17th century, a house was built by the Lennox family adjoining the northwest end of the gatehouse.

The house was sketched in 1725 by Samuel Buck and is depicted as a large two-storied building with gabled dormer windows set into a steeply pitched roof.

[5][12] The castle eventually returned to the possession of the Crown,[9] but fell into disrepair, and by 1600 the building was described as "old and ruinous".

[13] The Bruce family retained the castle and manor until the late 19th century, when they were sold to James Emerson of Easby Hall.

[14] The castle is currently privately owned, having been bought by Osbert Peake, 1st Viscount Ingleby, in the mid-20th century as part of a shooting estate.

[20] The entrances would originally have been blocked by portcullises made of wood or metal that could be raised or lowered by winches set into the gatehouse walls.

A vice or spiral staircase enclosed by a tower projecting out from the northwest wall gave access to the upper floors.

The area immediately adjoining the castle was landscaped during the late medieval period, when ornamental gardens were built in two rectangular enclosures a short way to the east of the bailey.

[10] The landscape and the site of the deserted village of Whorlton are included with the castle as part of a scheduled ancient monument.

A report produced for the park authority in 2005 examined several options, including retaining the building as ruins but improving security by employing a custodian, converting the gatehouse into a liveable property for use as a house or holiday home, or establishing a local community group to help manage and maintain the site.

The report concluded that "retention of the status quo is not felt to be an acceptable option, due to continuing damage through vandalism to the historic fabric and archaeology, together with the impaired public enjoyment and the uneconomic nature of future repairs and maintenance to the site.

"[22] English Heritage part-funded a feasibility study to assess the prospects of turning the gatehouse into a home.