Mayor William Holmes Howland and the press supported the workers' right to organize but Smith refused to allow a union and criticized the city government for failing to maintain order.

Smith's actions alienated the Toronto working class, particularly Catholic workers among whom he hoped to build a political base.

Due to age and poor health he declined to continue in cabinet when Sir John Sparrow David Thompson became prime minister in 1892.

He agreed, however, to return to cabinet as a minister without portfolio under Mackenzie Bowell and then Sir Charles Tupper until the Conservatives were defeated in the 1896 federal election.

In 1872, he lobbied Macdonald for the gradual release of Fenian raiders who had attempted military attacks on Canadian soil in order to promote Irish independence.