A solicitor, businessman and citizen soldier before his entry into politics, Gould was a member of the New South Wales Legislative Assembly from 1882 to 1898, during which time he served as Minister for Justice in two Free Trade governments.

Gould's interest in parliamentary procedure saw him become involved with the relevant standing committee and he was elected unopposed as the second President of the Senate in 1907.

[1] By 1887 he was supporting his career with his own legal practice in Singleton and Sydney, and when the party system came into being in New South Wales in that year he identified as a Free Trader.

[3] During the first period of his ministry he became involved in a dispute with the Chief Justice, Sir Frederick Darley, over requests for improved court accommodation and a contested punishment for contempt of two witnesses.

Despite embarrassing his premier, he remained close with Parkes throughout his state career and in later years frequently gave speeches in his memory.

[3] Gould was also involved in defence matters, supporting the introduction of conscription and maintaining loyalty to the Empire, where "there are men much more experienced in the principles of government than we are".

[3] A supporter of the White Australia policy, Gould expressed concern over Kanaka labour in Queensland, although his assumption that the arrangement was temporary enabled his pragmatism on this issue.

He had supported the opening of Senate proceedings with prayer and had made frequent reference to the standing orders of the British House of Commons.

He endeavoured to keep the Senate representative of the states' interests and free from party politics, and his rulings focused on unparliamentary language and relevance.