Despenser first played an important part in 1258, when he was prominent on the baronial side in the Mad Parliament of Oxford.

In 1260 the barons chose him to succeed Hugh Bigod as Justiciar, and in 1263 the king was further compelled to put the Tower of London in his hands.

He was the son of Hugh le Despenser and was summoned to Parliament by Simon de Montfort.

By his wife, Aline Basset, he was father of Hugh Despenser 'the elder', who became an advisor to Edward II and was made Earl of Winchester.

They also had a daughter named Eleanor le Despenser, who married Sir Hugh de Courtenay, feudal baron of Okehampton.