This early 11th-century Norman motte-and-bailey fortification was built by Bernard de Neufmarche, the half-brother of William the Conqueror.

[1] It is thought to have fallen to a Welsh attack sometime between 1121 and 1136, and after short abandonment was possibly rebuilt by Walter Clifford in the 1150s, with Edward I of England spending three days in Trecastle quelling a revolt in 1295.

At 6.6 m high it is the largest motte and bailey the Brecon Beacons National Park (Welsh: Parc Cenedlaethol Bannau Brycheiniog).

The village takes its name from the old motte and bailey, whose original name was 'una villa nostra de Lliwel', although to this day Llywel is the name of the neighbouring hamlet, local parish and electoral ward.

The local name Trecastle emerged by the end of the Medieval period, meaning "the town of the castle".