He was born in Wollongong to storekeeper Richard Millard and Anne, née Radler.
In 1854 his family moved to Ulladulla, and he was educated locally before entering his father's tanning business.
[2] He continued to serve the region for 27 years, through 4 districts and 3 political parties.
[4] For the 1920 election the Legislative Assembly was elected using a form of proportional representation with multi-member seats and a single transferable vote (modified Hare-Clark) and Bega was absorbed by Goulburn.
James resigned just six months later to accept an appointment as an Acting Judge of the Supreme Court of New South Wales,[5] and Millard, being the unsuccessful Nationalist candidate, was appointed to the vacancy, taking his seat on 15 December 1920.