The son of an officer in the East India Company, he was born in Banner Street, St. Luke's, London, on 21 February 1796.

Samuel Oughton was sent to Jamaica from Sherman's Surrey Chapel in 1836; an arrangement on behalf of the Baptist Missionary Society, a body that worked closely with the LMS, and practiced congregational principles of church governance.

Shortly before 1840, James Sherman became a founding trustee and director of the Congregationalist's new non-denominational enterprise - Abney Park Cemetery.

This was Uncle Tom's Cabin, by the American Congregationalist Harriet Beecher Stowe, with Sherman's introduction from London.

For the book's promotional tour in London in the early summer of 1852, Stowe, her husband, and her brother Charles Beecher, stayed at Sherman's house.

At the same time he invited the African-American escaped slave and Congregational minister Samuel Ringgold Ward, and assisted his stay in Britain for nearly a year, helping him raise funds for the Canadian Anti-slavery Society at a time when many escaped slaves from the USA were trying to reach freedom in British Canada.

Sherman became minister at Blackheath Congregational Church 1854-62 and was succeeded at Surrey Chapel by Christopher Newman Hall.