There he remained, receiving a rudimentary education, including how to read but not how to write, until he was indentured at the age of 10 to a carver and gilder in Great Portland Street.
Tried and approved as a preacher there, he admired William Huntington's high Calvinism, though it is unclear that this led, as his detractors claimed, to his practising Huntingdon's "practical antinomianism" or showing wanton disregard for accepted Christian morality.
In the course of 1813 rumours began to spread in the Weekly Dispatch and other pamphlets and broadsides connecting Church with the White Swan (a well-known homosexual brothel or 'gay bar' in modern parlance, in Vere Street, Clare Market), saying that Church was its chaplain and had performed mock marriage ceremonies for its male customers subsequently recognised by some modern historians as same sex marriages.
Church denied this connection with Vere Street, claiming that it was propaganda by his clerical opponents and successfully taking legal action to prevent the Dispatch from publishing further reports.
During the 730 days he served of his sentence at Newington and Horsemonger Lane gaols, he was often in great anguish according to his autobiography, but he received many visitors, had access to books, retained many of his followers (especially women) and had his four children cared for.