Ivo became a figure of some derision when, during the Siege of Antioch, he and some his compatriots (including his brother) panicked, let themselves over the city walls by rope and fled back home.
He was among the barons supporting the claims of Robert Curthose against his brother Henry I in 1100.
[2] In 1102, after the attempt to put Curthose on the English throne had failed, Ivo was severely fined by King Henry I for waging private war against his neighbours.
In addition, Robert agreed to eventually marry the infant daughter of his brother Henry de Beaumont, 1st Earl of Warwick to Ivo's young son, and to return the mortgaged lands to the son.
Two years later Henry I made plans to return the lands to the sons, who were probably serving in the king's household, but both died in the shipwreck of the White Ship in 1120.