He was defeated by the Labor candidate David McKenzie in 1972, and returned to his legal practice, focusing primarily on industrial law.

[4] He was briefly appointed Minister for Business and Consumer Affairs following the forced resignation of John Moore in April 1982.

His major act in that portfolio was the announcement of an inquiry into the "colour-television affair" which had led to the resignations of Moore and Michael MacKellar.

He held that position until the Fraser government was defeated at the 1983 election; he again lost the seat of Diamond Valley to a Labor candidate, in this case, Peter Staples.

[6] At the early 1984 election, Brown re-entered parliament as the member for the newly created Division of Menzies, essentially the more Liberal-friendly eastern portion of his old seat.

[8] Brown served as Howard's deputy until July 1987, when a leadership spill was called in the aftermath of the party's defeat at the 1987 election.

[9] Peter Shack, who was an unsuccessful candidate to replace Brown as deputy, made a criticism of Brown as he stated that coming after the Liberals' defeat at the recent 1987 election, its third consecutive defeat, the Liberal Party needed a deputy leader who could give "visible and effective political and parliamentary support to the leader".

[10] Brown remained a Coalition frontbencher – under Howard, Peacock, and John Hewson – until his unexpected resignation from parliament in February 1991.

[13] As of 2006, he was a mediator and arbitrator in domestic and international commercial, trade and construction matters and in Internet domain name disputes.