[1][2] The mound, situated a quarter mile from the river, was separated from the town by flat swampy land and was seen as a good site for a fortification.

The remains of the motte and bailey castle are located in Thornes Park, in the centre of Wakefield overlooking the River Calder.

[5] William de Warenne, 3rd Earl of Surrey probably started to build the castle, an earthwork motte and bailey structure during the Anarchy, the civil war between Stephen and Matilda.

This story was retold by the antiquarian, John Leland in his book, "The Itinerary of John Leland", a recollection of his travels around England between 1535 and 1543:[6] A quarter of a mile withowte Wakefeld apperith an hille of erth caste up, wher sum say that one of Erles Warines began to build, and as fast as he buildid violence of winde defacid the work.

[8] In July 1558 a beacon was lit on Lowe Hill to warn the population that the Spanish Armada had been sighted off The Lizard in Cornwall.