[1] He made his debut in a Southern League match against West Ham United which Stoke lost 5–0.
[1] He became known for his no-nonsense tackling and solid performances at left back and helped the club to gain a return to the Football League in 1914–15.
[1] However it was delayed due to the outbreak of World War I during which time Milne returned to his home of Hebburn.
[1] The league returned in 1919 and Milne established himself in the starting eleven and played in every match during the 1920–21 season and helped the side gain promotion to the First Division in 1921–22.
[1] He remained at the Victoria Ground for two more seasons before leaving for Doncaster Rovers after making 276 appearances in all competitions for the "Potters".