[10][11] The island is sometimes known as "Paddy's Milestone",[6][12] being approximately the halfway point of the sea journey from Belfast to Glasgow, a traditional route of emigration for many Irish labourers going to Scotland to seek work.

Its prominence is due to the microgranite's hardness, making it more resistant to erosion than the surrounding Permian and Triassic sedimentary rocks into which it was intruded.

[17] Boulders of distinctive Ailsa Craig microgranite known as erratics were transported by glaciers as far afield as County Donegal and Pembrokeshire.

The only surviving buildings on the island are the lighthouse on its east coast facing the Scottish mainland, a ruined towerhouse built by Clan Hamilton to protect the area from Philip II of Spain in the 16th century,[20] and the old quarry manager's house that is used by the RSPB.

The 'Horse Well' was located behind the gasworks; the 'Castle Well' stands above Ailsa Castle, and finally the Garry Loch sits higher up and once supplied water to the tenant's cottage.

[21] Fishermen seem to have used the island for centuries, first being noted in 1549[22] and it is recorded that they even at one time slept beneath sails stretched over hollows on the beach.

[31][32] Four cottages, a shed, a railway, a winch house, the main pier, and an area of adjacent land are now in the ownership of the Scottish Indian businessman Bobby Sandhu.

[35] Margaret Girvan ran a tearoom[36] in a wooden building that stood next to the tacksman's cottage,[37] famed for its pristine white table cloths and fresh scones.

[45] Another version states that Andrew Knox lay in wait for Hugh with nineteen others and ambushed him at the shingle beach with the result that he attempted to defend himself until he was forced back into the sea and drowned.

[46] Beneath the Main Craig at the southern end of the island and 12 metres (39 ft) above sea level is a cave named after the supposed smuggler MacNall.

[51] The monks of Crossraguel Abbey once held the island and "places of prayer" are therefore to be expected especially with a garrisoned castle nearby; it is to be noted that even the diminutive Lady Isle off Troon once had a chapel.

[55] The gasworks are still a prominent feature on the island and the cable-powered tramway was partly built to haul wagons full of coal up to it from the North Port.

The Northern Lighthouse Board's tramway had a section worked via a powered cableway that hauled wagons up from the North Port and later from the new jetty.

[60] This crudely constructed narrow gauge line was mainly horse drawn, although wagons were also moved by hand or hauled up inclines by winding engines.

[60] Wagons or bogies were winched up to the substantial stone crusher and gravity was used to deliver the different grades of road stone to the waggons below that were then hauled by horses to the Quarry Pier via a line that ran in front of the lighthouse buildings and took a tight right-angled bend to run up the substantial stone-built incline to the storage area in preparation for delivery via sea to the mainland.

Photographs taken in the late 19th century show the horse-drawn wagons passing in front of the lighthouse and portray the substantial railway incline and storage area.

At times the production outstripped the storage capacity and a photograph shows at least three piles of different grades of road stone stockpiled in front of the lighthouse enclosure.

From the mid-nineteenth century the island has been quarried for its rare type of micro-granite with riebeckite (known as "Ailsite"), which is used to make stones for the sport of curling.

[70] Slowworms (Anguis fragilis) are found on the island, although they suffered greatly when badgers and raccoons were introduced by Lord Ailsa.

The billy goats were shot for sport in the 19th century and no longer survive; only a mounted head of one remains at the McKechnie Institute in Girvan.

[72] The rabbits and goats may have been originally introduced to supply food for the fishermen and were mentioned by Pennant in 1772[36] and by the Rev Abercummie in 1688, who called them by the old name of coneys.

After a long campaign using pioneering techniques, the rats were eradicated in 1991, and now puffins are once again raising young on the island with many other benefits accruing to both the fauna and the flora.

[78] Lawson in the 1890s records that a young lady once fell over the cliff near Craig Na'an; however, her Victorian style garments caught the wind like a parachute and she escaped with her life and some broken bones that soon knitted back together.

A shocking death was that of a young boy from Girvan who was sitting amongst loose rocks, pulling out stones and throwing them into the sea when a very large boulder started to move and crushed him.