Sir James was the only surviving son of Patrick Galloway, a prominent minister from Perth, and his first wife Martillo Guthrie.

He married an unknown daughter of a knight named Sir Robert Norter, and by her had one son Thomas, the second Lord Dunkeld.

Mary became a nun at the convent of Val-de-Grâce in Paris, while James, who would have been the fourth Lord Dunkeld had the title not been forfeit, joined the military.

He eventually reached the rank of brigadier of infantry in the Garde du Corps, and was created a chevalier of the Order of Saint Louis.

and Azure, a Mound bespread with the Rayes of the Sun proper, embrac'd betwixt two Corn ears Saltoir wayes Or; and above all in an Escrol this motto, Higher.