During the 13th century, it was restored by Odon IV of Ham who gave the fortress its definitive shape, a polygonal enceinte broken up by large cylindrical towers.

Enguerrand de Coucy bought the seigneury in 1380 and his daughter sold it in 1400 to Louis d' Orléans, who integrated it into his network of fortresses which included La Ferté-Milon, Pierrefonds and Fère-en-Tardenois.

United with the French crown under the reign of Henri IV, it was transformed at the end of the 17th century by Vauban.

It 'welcomed' many famous prisoners, the last of whom was Prince Louis-Napoléon Bonaparte (the future Napoléon III) who, after six years, escaped by adopting the identity of a painter, Badinguet.

Later, his opponents would often refer to him disparagingly as Badinguet In 1870, during the Franco-Prussian War, the Second Army of the North encircled the town of Ham and forced the occupying Prussians to sign a surrender.