She was the daughter of engineer and later director of AEG Georg Stern and his wife Lisbeth (née Schmidt), who was the younger sister of artist Käthe Kollwitz.

Hofer's younger sisters were dancer Katta Sterna and actress Maria Matray.

Hofer's father was Jewish and her mother was Lutheran.

In 1932, Hofer and her husband, Austrian actor Fritz Kortner, left Germany and lived in Switzerland, then Austria and the United Kingdom, before settling in the United States in 1938.

In 1941, the couple moved from New York City to Los Angeles.