He was the founder of the Free Church of Tonga and enjoyed significant influence during the reign of George Tupou I, who made him prime minister.

A disagreement arose with the Wesleyan authorities at Sydney in 1879, and Baker founded an independent body under the title of the "Free Church of Tonga".

In late 1886, a group of chiefs from Muʻa – led by Siaosi Tukuʻaho – hatched a plan to arrest Baker and deport him to Fiji, likely with the support of Basil Thomson.

The first attempt was made on Christmas Eve, when Lavuso – a Muʻa supporter and staff member of Tupou College – and another man went to Baker's house intending to club him to death, but were distracted by the presence of a group of schoolchildren.

[4] On 12 January 1887, the four outlaws and Tōpui, the son of a Muʻa chief, arrived in Nukuʻalofa and hid along Beach Road near Baker's house.

With no opportunity arising to ambush Baker, they stayed with Lavuso at Tupou College before returning to Beach Road the following evening.

[5] King Tupou regarded the shooting as an act of rebellion and called on the chiefs of Tongatapu to assemble armed forces for an expedition against the Muʻa conspirators.

[10] After his deportation, Baker visited the United States and attempted to lobby the American government to intervene in Tonga on his behalf.

However, his efforts were hindered by publication in November 1890 of a report by Basil Thomson into Tonga's finances, which showed a number of irregularities in the Treasury Department and that Baker had used government funds to pay for groceries, alcohol and other personal expenses.

Baker subsequently settled in Auckland where he "owned a large house in a select part of the town" and reportedly had "extensive investments".

R. L. Stevenson, who called Baker "the defamed and much-accused man of Tonga" found him "highly interesting to speak to" (Vailima Letters, p. 41).