Gurney joined with his friend, Joseph Fox, in 1795 and opened a Sunday school at Walworth, of which he in the following year became the secretary.
On 13 July 1803, he was present at a public meeting in Surrey Chapel schoolroom, when the "Sunday School Union" was established.
He was a member of the first committee of the London Female Penitentiary, formed in 1807, and was one of the lay preachers who for many years took the Sunday services in that institution.
In 1812, on the establishment of the Westminster auxiliary of the British and Foreign Bible Society, he was elected a member of the first committee, and soon after became secretary.
Towards rebuilding chapels in Jamaica and sending additional ministers there, he was a liberal contributor, besides frequently receiving Baptist missionaries in his own house.
The house was then licensed as a place of worship, and during four years Carey and other ministers held Sunday evening services in the drawing-room.
Gurney died at Denmark Hill on 25 March 1855 and was buried in the family vault at West Norwood Cemetery.