James helped his father on the farm until manhood and then sought a wider and more financially rewarding direction for his future, first as a commission agent and dealer in the grain trade.

By the 1871 Census, James had moved to Aberlour,[5][full citation needed] and in 1874 established and developed a new branch agency for the North of Scotland Bank.

[6] He became a prominent and respected member of the local community; as an elder of the parish church and in several public positions: chairman of the School Board, county councillor, and town Provost.

[7] In 1879, Fleming secured a feu from the Earl of Fife on the land where he planned to build the Aberlour distillery and, upon its completion, produced whisky there from December 1880.

In James Thomson's 1902 book Recollections of a Speyside Parish,[12] the author wrote: "The hospital that has been built and endowed by his [Fleming's] liberality will always remain a memorial to his Christian philanthropy.