[11] As ordered by Archbishop Alexander de Bicknor, the initial castle was built between 1324 and the 1340s, to defend the settlement.

[11][3][10][12] The original castle is thought to have comprised high walls with a courtyard in the centre.

[3][10][12] In 1822 the property was sold to Major Palmer, Inspector General of Prisons, who pulled most of the palace down and used the materials to build his mansion, 'Tallaght House', as well as a schoolhouse and several cottages.

A tower from the original castle was left untouched and later was incorporated into the current priory building.

[3][10] Major Palmer later sold the mansion and lands to his successor as Inspector of Prisons, Mr Lentaigne.