Milstein was born in Odessa, Russian Empire, the fourth child of seven, to a middle-class Jewish family with no musical background.

Auer did not name Milstein in his memoirs but mentions "two boys from Odessa ... both of whom disappeared after I left St. Petersburg in June 1917.

"[4] Milstein met Vladimir Horowitz and his pianist sister Regina in 1921 when he played a recital in Kiev.

"[5] Milstein and Horowitz performed together, as "children of the revolution", throughout the Soviet Union and struck up a lifelong friendship.

A transcriber and composer, Milstein arranged many works for violin and wrote his own cadenzas for many concertos.

After playing many different violins in his earlier days, Milstein finally acquired the 1716 "Goldman" Stradivarius in 1945 which he used for the rest of his life.

A recital he gave in Stockholm in June 1986, one of his last performances, was recorded in its entirety[7] and shows the remarkable condition of his technique at age 82.

During the late 1980s, Milstein published his memoirs, From Russia to the West, in which he discussed his life of constant performance and socializing.

He also discusses his best friends, pianist Vladimir Horowitz, cellist Gregor Piatigorsky and ballet director George Balanchine, as well as other violinists such as Fritz Kreisler and David Oistrakh.