He left school to go to work for a local stable and eventually made his way to Toronto, Ontario where he was introduced to Thoroughbred racing.
While he started his new career slowly, after earning his first win on June 29, 1967 he soon became one of the top apprentice jockeys on the Ontario circuit.
Frequently among the top jockeys in wins during the 1970s and 1980s at Greenwood Raceway, and at Woodbine and Fort Erie Racetracks, he also traveled to compete around the world.
[citation needed] Notable among the horses Duffy rode were two Hall of Fame inductees.
[3] Retired after thirty years as a jockey, on October 17, 1998, in an event that raised money for charity at the racetrack in his native Charlottetown, Duffy was the celebrity guest jockey who rode in a "man vs horse" match race against sprinter Ben Johnson.