The house and a core demesne of 120 acres were bought by a group of people looking to preserve them, and became the first major project of the Irish Georgian Society; they were later transferred to a dedicated charitable foundation, and ultimately to State ownership.

Most of the wider estate remaining was divided between State forestry company, Coillte, and developers, and parts were built on, notably the former orchard and walled garden.

The house was entered by ascending a staircase outside before coming into a large entrance hall which was decorated with stucco gilding and pictures of the family.

On William James' death in 1754 it passed to his son Tom Conolly whose wife, Lady Louisa (great-granddaughter of Charles II of England and Louise de Keroualle), finished the interior decoration during the 1760s and 1770s.

Lady Louisa also ordered extensive work on the grounds; the drainage scheme through the woodland is ingenious, creating dry paths for walking on land that is below the watertable (the ha-ha fence is part of this intricate network).

[4] Tom's eldest son Thomas inherited Castletown, but after he was killed in 1900 the estate passed to his brother Major Edward Conolly.

The Conolly family continued to live in their ancestral house, latterly as the Conolly-Carews, until 1965, when it was sold by the 6th Baron Carew, along with its collections and land, to a Major Wilson for £133,000.

The house was bought in 1967 by Mariga and Desmond Guinness for £93,000 to save it from vandalism, became the flagship of the Irish Georgian Society, and was eventually handed over to the newly established Castletown Foundation.

However, some other features, such as the Gazebo, the Steward's House and a mock temple, were retained, after negotiations between developer Janus Securities and the Irish Georgian Society.

In September 2023, it was announced that the vehicular entrance, and both main and disabled car parking areas, would be closing, due to disagreement between the State agency managing Castletown and the new owners of much of the remaining land, leading to active local protests.