After the execution of the 3rd Duke in 1521, and his posthumous attainder, the castle and manor of Stafford escheated to the crown, and all the peerage titles were forfeited.
His descendants, much reduced in wealth and prestige, retained possession of Stafford Castle and the widow of the 4th Baron was still seated there during the Civil War when shortly after 1643 it was destroyed by Parliamentarian forces.
[1] On his death the following year, unmarried and without issue, the senior male line of the Stafford family was extinguished.
The descent of the feudal barons of Stafford is recorded, amongst other places, in the Rimed Chronicle of Stone Priory, a verse of unknown date which was found inscribed on a tablet hanging at Stone Priory (founded by the 1st feudal baron) in 1537 at the time of the Dissolution of the Monasteries and transcribed and printed by Dugdale in his Monasticon Anglicanum.
[2] Modern standard sources (which largely agree with it) state the descent as follows:[3] The first feudal baron held 131 manors as listed in the Domesday Book of 1086, a high proportion lying in Staffordshire.