In 1446, when he travelled to Calais and Brussels, he was painted by Petrus Christus, an Early Netherlandish painter active in Bruges.
[2] His first wife died sometime before 1456 when he remarried Mary Drury, a woman from Suffolk, daughter of Sir William Drury and through her mother a great-granddaughter of Katherine Swynford.
After Mary Drury had died in 1469, Grimston married a third time with Philippa, the widow of Lord Roos.
[2] He died in 1478 and is buried in the church of Thorndon, Suffolk next to his second wife Mary.
[2] His portrait has remained in the hands of his descendants the Earls of Verulam, currently John Grimston, 7th Earl of Verulam, but is on long-term loan to the National Gallery.