In 1666 he inherited the peerage of Baron Willoughby of Parham, and from 1667 he served as Governor of Barbados (1667 – 1673).
[1] From 1667 until his death he was Governor of Barbados, revisiting England occasionally and retaining his other offices.
Willoughby died in Barbados in 1673, and his body was brought back to Lincolnshire, to be buried at Knaith.
[2] Willoughby married Anne, daughter of Philip Cary (MP for Woodstock) of Aldenham, Hertfordshire, by 1637.
[2] They had a daughter, Frances, who married Charles Kirkhoven, 1st Earl of Bellomont.