In 1809, on becoming the perpetual curate of the chapel, he made over to trustees a fee-farm rent of £21 a year as an endowment; he also erected at his own expense a minister's residence in Well Street.
It was rumoured, baselessly that during Lord Liverpool's long premiership every see that fell vacant was offered to Norris, with the request that if he would not take it himself, he would recommend someone else; so he had the nickname "the Bishop-maker".
From 1793 to 1834, as a member of the committee of the Society for Promoting Christian Knowledge, he largely ruled its proceedings; but in 1834 there was a revolt against his management, and he was left in a minority.
Norris's best known work is A Practical Exposition of the Tendency and Proceedings of the British and Foreign Bible Society, in a Correspondence between the Rev.
This correspondence arose from an attempt made by James William Freshfield (1801–1857) to form an Auxiliary Bible Society in Hackney, to which Norris strongly objected.