Originally a 12th-century fortress, it was transformed in the 15th century by Foulques d'Agoult, chamberlain of King Rene I. of Anjou.
Afterwards the castle became the residence of the Créqui-Lesdiguières family, who were the lords of Château de La Tour d'Aigues.
Despite owning the castle, they never took residence in it, and this state of affairs continued until the beginning of the French Revolution.
Finally, in 1920, Robert Laurent-Vibert, a producer of cosmetics, bought the ruined castle and employed the architect Henri Pacon to restore it.
In his will he donated the castle to the Académie des Sciences, Agriculture, Arts et Belles Lettres under condition that it should be transformed into a trust, which would support young artists.