[2] On taking Holy Orders (deacon, 1848, priest 1849),[3] he became curate of Banningham, Norfolk, and then of Christ Church, Tunbridge Wells.

He was called to the Rectory of Hinton Martell in 1852 and to the Vicarage of Christ Church, Hampstead in 1855,[3] a position in which he remained for 30 years.

1891 he travelled to Japan on a visit to the mission churches of the Nippon Sei Ko Kai, reuniting with his eldest son Edward, then serving as the first Bishop of South Tokyo.

Arriving in Yokohama on 23 September 1891,[4] the travel journals of his daughter, Mary Jane Bickersteth,[5] include detailed descriptions of the Anglican church's mission work in Japan as well as visits to sites such as the Shrines and Temples of Nikkō and the experience of surviving the strong Mino–Owari earthquake at Osaka on 28 October 1891.

His Hymnal Companion called forth from Dr. Julian, editor of A Dictionary of Hymnology, these high words of praise: "Of its kind and from its theological standpoint, as an evangelical hymn book, it is in poetic grace, literary excellence, and lyric beauty, the finest collection in the Anglican Church;" and the author's contributions to this volume are pronounced "very beautiful and of much value."