Sherard was born in Bushby, Leicestershire and studied at St John's College, Oxford, from 1677 to 1683.
In 1690 he was in Ireland as tutor to the family of Sir Arthur Rawdon at Moira, County Down.
When he returned to England he became a patron of other naturalists, including Johann Jacob Dillenius, Pietro Antonio Micheli, Paolo Boccone and Mark Catesby.
He was also instrumental in the publication of Sebastien Vaillant's Botanicon parisiense (1727) and Hermann's Musaeum zeylanica.
Dillenius's famous Hortus Elthamensis, which was often cited by Linnaeus was a description of the rare plants that James Sherard grew in his garden in Eltham in Kent (now within the confines of Greater London).