He became count of Hainaut after the death of his older brother, Arnulf III of Flanders, at the battle of Cassel.

During Baldwin's minority reign, which lasted until 1083, Richilde constantly fought against Robert to recover Flanders for her son, but she was unsuccessful.

Their children were: Baldwin joined the First Crusade in the army of Godfrey of Bouillon (rather than with his nearer relative Robert II of Flanders, whose family was still at odds with his own), after selling some of his property to the Prince-Bishopric of Liège.

In 1098 he was sent back to Constantinople with Count Hugh of Vermandois after the siege of Antioch, to seek assistance from Byzantine emperor Alexius I.

While on a pilgrimage to Jerusalem in 1106, Baldwin's wife Ida organized a search for her lost husband in Anatolia, which was inconclusive.