[2] Recruited from Xavier College, Brushfield started out at Geelong as an 18-year-old in the 1962 VFL season, where he made two league appearances.
[5] On 6 July 1963 he was a member of the Geelong team that were comprehensively and unexpectedly beaten by Fitzroy, 9.13 (67) to 3.13 (31) in the 1963 Miracle Match.
Over the next two years, Brushfield made seven more league appearances, then left for the Victorian Football Association (VFA).
[7] He was made captain-coach in 1967 and took Geelong West to the Division 2 grand final, which they lost to Oakleigh by 13 points.
[8][9] In 1968, at the age of just 24, Brushfield led Geelong West to a premiership, with a grand final win over Williamstown.