[3] Returning to Melbourne the next year, Smith was admitted to the Victorian Bar and on 15 January 1879, married Sara Jane Creswell, who bore him four sons and three daughters.

Developing an interest in politics, Smith unsuccessfully stood for the Victorian electoral district of Emerald Hill in February 1880, before moving to Sydney in 1881, where he won a Legislative Assembly by-election for Gundagai in 1882.

Disinherited by his father, Smith returned to Sydney to continue his career as a barrister and founded the New South Wales Employers' Union.

Elected as the member for Glebe in the New South Wales Legislative Assembly in February 1889, Smith was almost immediately promoted by Premier Henry Parkes to Secretary for Public Works, and later, Treasurer.

Smith proved to be a hard working minister but abrasive figure, frequently clashing with Parkes and accused of threatening to "shoot down" striking maritime workers "like bloody dogs".

Smith lost Nationalist preselection at the 1919 election and was defeated as an independent candidate, having spent almost all his period in federal parliament in opposition.

[10] According to Geoffrey Bolton, Smith was once "dismissed as a spokesman for employers who wanted to introduce cheap non-European labour into White Australia", but by the late 20th century had been rehabilitated as "a tolerant multiculturalist ahead of his time".