Rose was elected to the House of Commons of Canada for the electoral district of Huntingdon, Quebec on 20 September 1867, and was later appointed Minister of Finance in the government of John A. Macdonald.

In 1869, Rose moved to England to practise law and acted as an unofficial representative of the Canadian government.

In this function, he answered directly to the Prime Minister, John A. Macdonald, since it was an informal position and was therefore not under the purview of the Governor General, meaning that communications could be kept secret from the Imperial Government.

He also sat on a number of Royal Commissions in Britain and became a baronet in 1872, and a member of the Imperial Privy Council in 1886.

Following his passing in 1888, Sir John Rose was interred at Mount Cemetery in Guildford, near the Loseley Park estate, which he had rented for some years.

His daughter Mary Rose married Major-General Sir Stanley de Astel Calvert Clarke, K.C.V.O., C.M.G.

His Clarke granddaughters, Mrs. Edith Mary Bibby and Mrs. Baird, were two of the noted beauties of the day, and their portraits by Fildes, R.A. (former) and by Shannon (latter) were exhibited at the Royal Academy, 1896.