Born in Crook, County Durham; Milburn was interested in all sports and became well known for his running and football skills.
His running career was at a time of big bets, weighted body belts and lead in the pumps - all to bamboozle the bookies.
As a school child he played football for Crook District (10-14's) and the team always did well in competitions, travelling to many distant locations.
This schoolboy team drew larger crowds than Crook United, having local attendances in excess of 5–6000.
He was sent to Cairo for three weeks whilst waiting to be transferred elsewhere however the man in charge of the camp discovered his talent and he remained there for two years.