[2] Duffy spent his early years in Mildura and in Albury, New South Wales, where he attended the Christian Brothers' College.

[3] However, he was later identified as factionally unaligned but part of the informal "Victorian Independents" group within the ALP, along with John Button, Barry Jones and Jim Kennan.

He sought to maintain media diversity by preventing major television networks from securing licences in regional areas.

[8] In 1992, Duffy played a key role in overturning the Australian Defence Force's ban on openly gay members.

Although defence minister Robert Ray had initially announced the ban would continue, Duffy brought the matter before the cabinet as a whole, arguing that Australia was in breach of its obligations under international law.