In the Middle Ages, the spot was occupied by granaries.

After they burnt down in the last years of the 17th century, Peter the Great engaged a team of Russian and German architects to construct the Kremlin Arsenal, designed to be one of the largest buildings in Moscow at the time.

Construction started in 1702, but was interrupted due to lack of funds during the Great Northern War with Sweden, and was only completed in 1736, under supervision of Field-Marshal Burkhard Christoph von Münnich.

During Napoleon's invasion of Russia, the retreating French soldiers had the central part of the building blown up.

It was restored between 1816 and 1828 to a Neoclassical design in order to house a museum dedicated to the Russian victory over Napoleon.