After managing the fledgling Canada Publishing Company, Bain was selected in 1883 to become the first chief librarian of the Toronto Public Library.

In the campaign to establish a free library, some of the opposition had argued against it on the grounds that taxes shouldn't be used to circulate novels.

[3] In 1903, Bain secured Carnegie funding to build a new central library (now the Koffler Student Centre at the University of Toronto) and three branches, Yorkville, Queen and Lisgar, and Riverdale.

He was also a founding member of the Champlain Society and held the role of Treasurer and was the president of the Royal Canadian Institute from 1900 to 1902.

James Bain is buried in Mount Pleasant Cemetery, along with his wife, Jessie Mary Paterson, and some of their children.