Leroy was born in Chantilly, the son of Jean-Jacques Leroy, building inspector of the Prince of Condé, and Mary-Anne Dunu, daughter of the superintendent of the Château de Chantilly.
In 1761, he married Toudouze Françoise-Thérèse, daughter of the prince's master of the hunt.
He was appointed architect of the Château de Chantilly in 1768, upon the death of his predecessor, Brice Le Chauve.
In 1782 he revised Bélisard's plan for the Place du Palais Bourbon.
With the landscaper Lecourt, he created the picturesque gardens of Betz-en-Multien[1] for the Princess of Monaco, mistress of the Prince of Condé; all that remains is a prostyle Ionic temple of Love.