It is not known who originally built the castle but in 1146, it was captured by Cadell ap Gruffydd, Prince of Deheubarth, however he was incapacitated as ruler of Deheubarth five years later when he was attacked while out hunting by a Norman force from Tenby, and was left for dead.
At the time, Totnes Priory held St Peter's Church at Lampeter, and the original grantor may have been Stephen, constable of Cardigan.
Until the Statute of Wales in 1284, Lampeter was fiercely contested between the English and the Welsh because it was on a trade route that both sides wanted to control.
Although there would have been a bailey, its precise location is unknown, but its outline may have influenced the layout of nearby retaining walls, and may have extended towards the church.
The motte has been somewhat truncated by building work and incorporated into the landscaping of the college grounds.