He later joined the Condé Army of émigrés where he served as aide-de-camp to Marshal Charles Eugène Gabriel de La Croix.

During the Bourbon Restoration and became aide-de-camp and First Gentleman of the Chamber of the Count of Artois (later King Charles X), Colonel of the National Guard and Peer of France on 4 June 1814.

[11] He distinguished himself during the trial of Marshal Michel Ney, judged by the Chamber of Peers, by insisting on death penalty, the verdict which he brought to the Palais des Tuileries on 6 December 1815.

He also played a role in the trial brought against Gen. Henri Gatien Bertrand, his brother-in-law, by publishing a letter in which he claimed that the general had taken an oath to Louis XVIII.

[14] The Duke of Fitz-James died at the Château de La Rivière-Bourdet in Quevillon on 15 November 1838 and was succeeded in the dukedom by his son, Jacques.