[4] The first castle in Neath was located west of the river[1] near the Roman fort of Nidum, and was a timber fortification in a motte and bailey structure.
[5] When Richard de Grenville founded Neath Abbey close by, he abandoned this original castle, and it may have been used by the monks as a source of building material.
[10] It was taken again, and this time destroyed, by Humphrey de Bohun, 4th Earl of Hereford, in 1321, during the rebellion against King Edward II of England.
[11] Its owner, Hugh Despenser the Younger (who had gained the lordship through his marriage to the heiress Eleanor de Clare), rebuilt it, possibly with the addition of a gatehouse.
[10] In 1376, one Roger Kyngot was the constable,[12] and the castle was rebuilt in stone in 1377; that is probably when the great gatehouse was built, which is the main surviving feature.