Mayor William Holmes Howland and the press supported the workers' right to organize but Smith refused to allow a union and criticized the city government for failing to maintain order.

Due to age and poor health he declined to continue in cabinet when Sir John Sparrow David Thompson became prime minister in 1892.

Following Thompson's death in December 1892, Governor General Lord Aberdeen invited Smith to become prime minister but he declined.

He agreed, however, to return to cabinet as a minister without portfolio under Mackenzie Bowell and then Sir Charles Tupper until the Conservatives were defeated in the 1896 federal election.

In 1872, he lobbied Macdonald for the gradual release of Fenian raiders who had attempted military attacks on Canadian soil in order to promote Irish independence.