The collection was started by artist Antanas Žmuidzinavičius (1876–1966), and a memorial museum was established in his house after his death.

[1] In 1982, a three-story extension was built to house the expanding collection and, as of 2009, the museum's holdings had grown to 3,000 items.

For example, one sculpture depicts Hitler and Stalin as devils in a dance of death over a pile of human bones.

[1][3] The first floor of the museum contains Žmuidzinavičius's collection which he acquired despite Soviet law against religious artefacts.

[4] The third floor contains many devils from former Soviet territories[4] but others come from countries around the world including Japan, Cuba, and Mexico.