His father, a chemist, died when James was only four days old, leaving his widowed mother to bring up three sons alone.
After school, and without any formal higher education, Wilson entered the Civil Service after having passed their entrance exam.
It was around this time that he left the Civil Service to pursue a full-time career in music.
He had an interest in some less typical instruments, and wrote substantial works for the free-bass accordion, the cor anglais, and an electrically modified flute.
Wilson was a founder-member of Aosdána, an association of people in Ireland who have achieved distinction in the arts.