[2] The history of the museum starts in 1956, when the Royal Academy for Art and Design in 's-Hertogenbosch organized its first international exhibition.

This led to the creation of the Municipal Exhibition Service or Gemeentelijke Tentoonstellingsdienst, which was made independent in 1972.

The Kruithuis also came to house the 'Foundation for Visual Arts Loans', or Stichting Uitleen Beeldende Kunst.

In 2002 the municipal government decided to cancel the plans at the Kruithuis, and to look for a location in the city center.

This caused that from 2003 to 2013 the museum was located in an old commercial building in the Palace Quarter southwest of the Railway station.

The museum maintains the archives of Gijs Bakker, Marion Herbst and Emmy van Leersum.

The collection has jewelry by Arman, Jean Arp, César Baldaccini, Louise Bourgeois, Georges Braque, Alexander Calder, Jean Cocteau, André Derain, Max Ernst, Lucio Fontana, Meret Oppenheim Pablo Picasso, Man Ray, Rob Scholte en Carel Visser.