There seems to be two phases of ancient hillfort building on Carn Fadryn, followed by a medieval fortification of the summit.

The first period of fortification at Garn Fadryn dates from about 300 BC, when the summit and an area of some 12 acres (49,000 m2) were enclosed.

The third fort which strengthens the natural crag near the summit is thought to be "the castle of the sons of Owain", mentioned in 1188 as being newly built.

(Giraldi Cambrensis Opera (Rolls series, 1868), VI, p123 'dua castra lapidea de nova sita fuerunt; unum...Deutrait; alterum...in capite Lhein, quod erat filium Oenei, cui nomen Karnmadrun.')

[2] On a clear day, the view from Garn Fadryn takes in Anglesey, Snowdonia, most of Cardigan Bay and even the Wicklow Mountains in Ireland.