At the same time, southern France became a major haven for Jewish refugees attempting to flee to neutral countries, whether legally or illegally.

On December 10, 1940, the Ministry of Defense set aside 600 acres (2.4 km2) south of the camp to house people expelled from Germany.

In 1939, at the start of World War II, the camp became a military transit base, and in 1940 a refuge for Spanish refugees fleeing from Francoist Spain.

Gradually, the Joffre camp became a place of internment for families of gypsies, Jews and Spanish refugees.

With a capacity of 8000, before long the camp became overcrowded, families were separated, and conditions deteriorated greatly.

At five o'clock in the morning on August 26, 1942, the foreign Jews in the southern zone were rounded up and taken to the Centre national de rassemblement des Israélites at Rivesaltes.

Rivesaltes was, during that time, the camp where the Jews arrested in the so-called "free zone" were gathered, and from which many of them (about 1,700) were sent to Drancy itself.

While the military part of Rivesaltes camp resumed its original purpose, a new "guarded residence center" was established there on September 12, 1944.

Located chiefly in block Q, this center housed people interned under the Vichy regime's épuration ("purification") policy.

Families considered "irretrievable"—a term used by administrators at the time—were sent at the end of 1964 to the Saint-Maurice-l'Ardoise military camp in the Gard until 1975.

Several hundred more families who had employment but no housing were accommodated in a "civil village" in the Rivesaltes camp during the 1960s.

The next decade saw the bulk of this population moved to the HLM (rent-controlled housing) of Réart, built in the city of Rivesaltes to finalize the situation of these families.

In 1986 an administrative detention center was created, initially to detain Spanish nationals who had entered French territory illegally.

[14][20][21] This gave backing to Christian Bourquin, the new president of the General Council of the Pyrénées-Orientales, and his opposition to the destruction of the site.

[10] In November of the same year, the General Council of the Pyrénées-Orientales acquired F block on the site, covering about 42 hectares.

[24] In January 2006, Rudy Ricciotti won an architecture competition for the memorial project, which Robert Badinter agreed to sponsor.

[28][29] In 2020, 270 works by the Catalan artist Josep Bartolí, who was interned at the camp, were donated to the museum by his family.