Gaunson was a prominent member of the "Stonewall" party led by Graham Berry, which, after unparalleled agitation in Parliament and in the country, ultimately annihilated the followers of Sir James McCulloch at the general election in 1877.
However, on presenting himself for re-election after his acceptance of office, he was defeated in Ararat by the Hon William Wilson, and had to resign his position in the Ministry, which he had only held from 9 July to 2 August.
[citation needed] At the dawn of the 20th century, Gaunson became involved with John Wren, a businessman who ran an illegal tote in the working class suburb of Collingwood, Melbourne.
His alleged role in the Wren machine is referred to in the Frank Hardy novel, Power Without Glory, in which the character "Garside" is clearly based on Gaunson.
His contemporary, Alfred Deakin, described him as "endowed with a musical voice, good presence, fine flow of language, great quickness of mind, readiness of retort and a good deal of industry, ability and humour, ... only disqualified from marked successes by his utter instability, egregious egotism, want of consistency and violence of temper".