He was the son of Sir Christopher Willoughby (died c. 1498) and Margaret or Marjery Jenney (daughter of Sir William Jenney of Knodishall, Suffolk, Justice of the King's Bench).
Secondly, on 5 June 1516, he married María de Salinas, the Spanish-born lady-in-waiting to Catherine of Aragon.
[2] Grimsthorpe Castle was granted by Henry VIII to the de Eresby family on the occasion of Maria's marriage.
By his second wife, Willoughby had a daughter, Catherine, who succeeded him in the barony on his death in 1526.
When Princess Mary died in 1533, fourteen-year-old Catherine was hastily married to the Duke of Suffolk, her fiancé's father, who was thirty-five years her senior.