Varey began his speedway career competing in open meetings in 1928 before signing for the Belle Vue Aces who had joined the newly formed Northern League in 1929.

[5] Varey began succeeding quickly but had a reputation of being a hard, uncompromising rider which often led him to disciplinary problems with speedway authorities, confrontations and occasional scuffles with other riders and on several occasions needing police escorts from stadiums after on-track action upset the home fans at other tracks.

During the winter of 1929–1930, Varey rode in meetings in Argentina, where the local fans gave him the nickname of El Diablo Rojo (The Red Devil) which further established his tough image even further.

Varey restarted speedway at Sheffield in 1960 in the Provincial League and remained until ill health forced him to relinquish control.

Varey returned to Belle Vue as team manager following the death of Dent Oliver the previous year but despite ongoing health problems he still helped with track preparation at Sheffield.