He succeeded to the earldom in 1425, but spent much of the rest of his life attempting to recover his inheritance, which his grandfather, the 1st Earl, had settled on his second wife, Lady Joan Beaufort (d.13 November 1440), the legitimated daughter of John of Gaunt and the children he had had by her,[8] giving rise to the Neville–Neville feud.

[14] As noted, Westmorland was involved in an ongoing struggle (the Neville–Neville feud), sometimes violent, to regain his inheritance from his grandfather's second wife, Lady Joan Beaufort, and his half-uncle Richard Neville, 5th Earl of Salisbury, Cardinal Henry Beaufort, and Thomas Langley, Bishop of Durham.

Lady Joan Beaufort died in 1440, and eventually, a settlement was reached in 1443 which, according to Pollard, represented a 'crushing defeat' for Neville, who regained the barony of Raby but was forced to concede the rest of the disputed lands to Salisbury.

[15] Westmorland was appointed a Commissioner of Array in 1459 and 1461,[16] and is said to have led troops raised in his name on the Lancastrian side in Durham in November 1460, but otherwise took little part in the military campaigns or political affairs of the day, and according to Pollard had by this time 'succumbed to a mental disorder', and been placed under the guardianship of his brother, Sir Thomas Neville (died 1458).

[20] According to Pollard, it is unclear who, if anyone, became Westmorland's guardian after the death of his brother, Sir Thomas Neville; however surviving documents indicate that Richard, Duke of Gloucester, the future Richard III, acquired an interest in Westmorland's estates, and occasionally used Raby Castle as his own residence.