[5] He moved with his parents to the United States in 1952, while still a teenager, to live with his trapeze artist sister Sylvia, who had lost an arm in a traffic accident, going on to serve in the US Marine Corps from 1954–56, being recruited after boxing against them for the YMCA.
[5] He missed out on selection for the US Olympic team, being ruled ineligible on residence grounds, and after his term of service, he returned to London and turned professional.
[5][2][7] Managed by Sam Burns, Downes won his first two pro fights before a defeat to future world champion Dick Tiger.
[5] After building up a record of 16 wins and 3 defeats, Downes won the British middleweight title, vacated by Pat McAteer's retirement, by beating Phil Edwards on 30 September 1958 at the Harringay Arena, London.
"[4] Downes moved up to light heavyweight in 1963,[12] winning his first three fights at the weight before facing Willie Pastrano for the world title in Manchester on 30 November 1964.
One of his more prominent roles was in Roman Polanski's 1967 film The Fearless Vampire Killers, in which he played "Koukol", a hunchbacked servant.
Downes was awarded the British Empire Medal in the 2012 Birthday Honours, in recognition of his sporting achievements and charity work.