[3][4] Virkhaus attended elementary school at Tartu Teachers' Seminary, and showed musical talent at a young age.
[6] During World War II, Soviet forces occupied Estonia and began political repression of suspected dissidents.
One day in 1944, Virkhaus's father was warned by a former student that the Soviets were planning a mass deportation, and his family might be among those targeted.
[7][8] While taking refuge with his family throughout Europe, Taavo made time to study the violin, practicing repertoire that his father wrote down from memory.
[6][9] In 1966, after defending his doctorate, Virkhaus was named to the faculty of Eastman, where he taught conducting and served as Director of Music at the University of Rochester.
[12] In 1987, he conducted the Toronto Symphony Orchestra in a program of all Estonian composers, including works by Rudolf Tobias, Eduard Tubin, and Arvo Pärt.