In 1411, Olivier joined the Armagnacs in the Armagnac-Burgundian Civil War despite being married to Isabella, a daughter of John, Duke of Burgundy.
[2] Although his father and grandmother had abandoned the family's claim to Brittany in 1381, Margaret and her sons Olivier and Charles de Avaugour plotted to seize the duchy.
In 1420, they invited John V, Duke of Brittany to a party at Champtoceaux, where they captured him and threatened him with death if he did not abandon the ducal throne.
After holding the duke for five months, Olivier's brother John de L'Aigle, Duchess Joan of Brittany, and the Breton nobility negotiated his release in February 1421.
While there, he married his second wife, Joan, daughter of Simon IV de Lalaing, Lord of Quiévrain (d. 1467).