Although he converted to Protestantism in 1891, Rosé was still considered a Jew from 1933 onwards by the National Socialists, who had just come to power, and imposed appropriate restrictions.
After his wife Emma Marie Eleanor Rosé-Mahler (1875–1933), Gustav Mahler's youngest sister, died in the year of the Machtergreifung, Rosé was defenceless against the harassment and repressions of the Nazis.
After that, the once celebrated cellist had to move to the so-called ghetto house reserved for Jews in Weimar's Belvederer Allee 6.
On 20 September 1942, Rosé was deported from there to the Theresienstadt Ghetto, where he enjoyed a special position as a "celebrity prisoner" (like Mahler's brother-in-law, a preferred composer of Adolf Hitler).
The musician emeritus died there in the early hours of 24 January 1943 at the age of 83, the official cause of death being "enteritis intestinal catarrh".