It is widely assumed that the castle was built by William Marshal the elder in the time period between 1207 and 1213 which he spent in Ireland.

[4][5] However, there have been doubts that the castle in Carlow supported the traditional function of a keep, i.e. to serve as a refuge of last resort.

Instead it deliberately diverted from the contemporary standard in England and continental Europe, i.e. there is no towered curtain, no gate house, and no undivided great hall.

[9][11] The castle was handed over to the crown in 1306, granted in 1312 to Thomas Plantagenet, confiscated by the crown in 1537 as the landlords were absent, bought by the Earl of Thomond in 1616, changed hands multiple times until it was taken by Oliver Cromwell in 1650 but was later returned to the Earl of Thomond.

In 1814 the castle was widely destroyed in an attempt to create more space for the conversion into a lunatic asylum with the help of explosives.