He was educated in Prescott and Brockville, Ontario, and joined the North-West Mounted Police in 1892 in Regina, Saskatchewan.

Clarke's first attempt at municipal office took place during the February 1912 election, when he ran for the position of alderman on Edmonton City Council.

During this council term, he forged a temporary alliance with mayor William McNamara against those who wanted to drive prostitution and gambling out of the city.

Clarke was sympathetic to the strikers, and when the federal government's Royal Commission on Industrial Relations (the Mather Commission) stopped in Edmonton, he joined labour leaders in presenting a list of twenty-nine demands, including the right to collective bargaining, the eight-hour day, price controls, and workers' rights to run and hold public office.

He was re-elected in 1935 and 1936 (defeating future mayor and labour advocate Harry Dean Ainlay on this second occasion).

He ran in every municipal election until the end of his life (for mayor in 1938 and for alderman in 1939 and 1940) but never again held political office.

Joseph Clarke was a perennial candidate for the Legislative Assembly of Alberta, running provincially five times.

He finished fourth of eighteen candidates on the first count, which would have been sufficient to win one of the riding's five seats under a plurality voting system; however, the district used a single transferable vote system that allowed more generally popular candidates to pass him on subsequent counts.

Clarke's final provincial election was a 1937 by-election in Edmonton caused by the death of incumbent Liberal G.H.

He finished second, as Liberal leader Edward Leslie Gray retained the seat for his party, which was backed by the Conservatives under the so-called "Unity Movement".

He made his first bid for a seat in the Canadian House of Commons, running as a Conservative candidate in the a 1902 by-election in the Yukon electoral district.

He made his second bid while still living in the Yukon in the 1908 federal election and was once again defeated, this time by Frederick Congdon.

Clarke did not make another run for federal office until 13 years later, when he ran in Edmonton East for the Liberal Party of Canada in the 1921 federal election (he was a personal friend of then-Liberal leader and soon to be Prime Minister William Lyon Mackenzie King).