In his early days, he was thrice imprisoned in the Bastille: in 1711 at the instance of his stepfather, in 1716 in consequence of a duel, and in 1719 for his share in the Cellamare Conspiracy of Giulio Alberoni against Philippe II, Duke of Orléans, the regent for Louis XV of France.

The following decade during the War of the Austrian Succession he fought with distinction at Dettingen and Fontenoy, where he directed the grapeshot upon the British columns, and three years afterwards he made a brilliant defence of Genoa.

The French ministers were deeply unhappy with d'Estrees the commander of the Army of Westphalia which had orders to invade Hanover but was moving extremely slowly.

Instead Richelieu, fearing that his army was not in a condition to face battle, concluded the Convention of Klosterzeven which allowed the enemy to escape destruction.

He and his officers were suspected of having stolen three-quarters of the money he was supposed to raise in taxes from the occupied territories, as well as stealing the pay of his soldiers.

After Pompadour's death in 1764, his position in court was restored and he developed an amiable friendship with the King's last mistress, Madame du Barry.

This was because the new Queen, Marie Antoinette, disliked both Madame du Barry and Richelieu's nephew, the overly ambitious Duke of Aiguillon.

In 1734, as a result of the intrigues of Voltaire, he married Élisabeth Sophie of Lorraine, the daughter of Anne Marie Joseph, Count of Harcourt.

He was also the lover of the famous courtesan and novelist Claudine Guérin de Tencin, and of Marie Louise Élisabeth d'Orléans, Duchess of Berry, elder daughter of Philippe II, Duke of Orléans, a very ambitious and powerful young widow who was infamous at the time for her alleged large sexual appetite.