The still functioning Castle Rushen clock is a notable landmark in Castletown,[9] having been presented by Queen Elizabeth I of England in 1597, while she controlled the Island during a dispute.

[9] After its initial construction and use by the Norse-Gaelic rulers of the Isle of Man, the castle changed hands repeatedly between the Scots and the English.

The Isle of Man was transferred to Scotland the year after Magnús Óláfsson's death[10] as part of the Treaty of Perth, ending the 1263–1266 Scottish–Norwegian War.

[10] The Island was captured in five days, the only resistance occurring at Castle Rushen, which was defended by Dungal MacDouall, who had once been Lord of Galloway and betrayed Robert Bruce's brothers, Alexander and Thomas, to Edward I who promptly had them executed as traitors.

After a three-week siege, Robert captured the castle on 12 June 1313, gaining the Isle of Man as an outpost securing the approaches to western Scotland and the Hebrides.

Known as The Great Stanley (Manx: Yn Stanlagh Mooar), James established a secondary Royalist court at Castle Rushen before leaving to fight the Parliamentarians in England.

[14] In August 1651 James sailed with two frigates, bringing 300 Royalists from the Isle of Man to meet Charles II in Lancashire.

Following the restoration work, and the completion of the purpose-built Victoria Road Prison in 1891, the castle was transferred from the British Crown to the Isle of Man Government in 1929.

From 1765 fines were no longer used to support the castle's maintenance and heavy structural deterioration set in, exposing the prisoners to cold and poor weather.

After initial Tynwald opposition to the expense, a new site was chosen and the modern Victoria Road Prison opened in April 1891, having been designed by local Manx architect James Cowle.

[22] The last person executed in the castle grounds was John Kewish, convicted of patricide, who was hanged in the Debtors' Yard in the summer of 1872.

[22] One notable prisoner held for a time at Castle Rushen was the 1697–1755 Anglican Bishop of Sodor and Man, Thomas Wilson.

Wilson was imprisoned in 1722 after a conflict with the 1702–1736 Lord of Mann, James Stanley, over who was to have authority over ecclesiastical court cases in the diocese of Sodor and Man.

[24] The 19th-century Manx democracy activists Robert Fargher and James Brown both spent time in the Castle Rushen gaol as punishment for attacking the House of Keys as an undemocratic institution and demanding public elections in the Isle of Man.

[16] Castle Rushen is one of four Manx National Heritage sites in Castletown, the others being the Nautical Museum, The Old Grammar School and the Old House of Keys.

The 28 miles (45 km) Millennium Way long-distance footpath, which opened in 1979, the 1000th anniversary year of Tynwald, starts at Castle Rushen before heading towards the northern Manx town of Ramsey.