He was educated at Bolton School, and went on to study at the Royal Manchester College of Music (RMCM), where his violin professors were Henry Holst and Thomas Matthews.
He subsequently taught at the RMCM;[3] he also had freelance engagements, and led the Buxton Spa Orchestra.
[2] Milner was chosen in October 1958 by John Barbirolli, conductor of the Hallé, to lead the Manchester-based orchestra.
Barbirolli respected his opinions and musicianship, and during absences on conducting engagements he left Milner detailed instructions for rehearsing particular works.
With his third wife Diana Wanklyn, principal double-bass player in the Hallé, who survived him, he had three sons.