A Conservative and Unionist, Kemp was elected five times to the House of Commons of Canada as the Member of Parliament for electoral district of Toronto East.

Leaving home at 16, Kemp headed to Montreal and eventually gaining employment as a bookkeeper at a hardware store.

He and Cecilia moved to Toronto in 1885, where he entered into partnership with Thomas McDonald, owner of the struggling Dominion Tin and Stamping Works.

The brothers expanded their operations and opened plants in Montreal and Winnipeg, eventually reorganizing their business as the Sheet Metal Products Company of Canada Limited in 1911.

Kemp was elected five times to the House of Commons of Canada as the Member of Parliament for the Ontario electoral district of Toronto East.

He used his business connections to rally opposition to the Laurier government's Naval Service Act as well as its plans for reciprocity with the United States - a campaign to which he recruited prominent Ontario Liberals.

Kemp married Cecilia Amanda Wilson (1858–1924) in 1879 and had three daughters: Alice Irene (Mrs. Walter Scott Waldie), Hazel Beatrice (Mrs. Francis Chattan Stephens), and Florence Evelyn (Mrs. Albert Henry Proctor).