A nephew of Robert Guiscard through one of his sisters, he was the count of Conversano from 1072 and the lord of Brindisi and Nardò from 1070, until his death.
The identity of Geoffrey's parents is unknown, but it is believed that his mother was a daughter of Tancred of Hauteville and it has been speculated that his father was Armand of Mortain.
[citation needed] According to Goffredo Malaterra, Geoffrey conquered most of his lands with his own energy (sua strenuitate) and without the help of Robert Guiscard.
Geoffrey joined his cousins Abelard and Herman, his brother Robert, and Henry, Count of Monte Sant'Angelo in the rebellion of 1079–1082, while the Guiscard was away fighting the Byzantine Empire in the Balkans.
While he was besieging Oria, a city on the Via Appia between Brindisi and Taranto, Robert returned at Otranto and began to march towards him.