Entered in the men's downhill at the 1964 Winter Olympics in Innsbruck, Milne died of a head injury after he lost control during a training run at Patscherkofel and struck a tree at more than 60 miles per hour.

Three days prior, British luge racer Kazimierz Kay-Skrzypecki died from injuries sustained in a training run.

Hugh Weir reported to the Australian Olympic Federation that Dr Blaxland said that he was wrong about his age (Milne was nineteen),[7] and that the IOC was wrong to suggest he was inexperienced: Manager John Wagner said that Milne had found the path 150 metres (500 ft) ahead of him obscured by contestants congregating because the top part of the downhill course was overcrowded, and tried to slow down "on a spot which was not prepared for stopping or swinging".

Following Milne's death and a serious injury to Edmund Schaedler of Liechtenstein,[3] some minor safety improvements were made to the downhill course prior to the race on 30 January.

He became the first non-European to win a men's World Cup downhill race in December 1969, held at Val-d'Isère, France.