Murray was the third son of John Murray, 1st Marquess of Atholl, by Lady Amelia Anne Sophia, daughter of James Stanley, 7th Earl of Derby, and was born at Knowsley Hall, Lancashire, the seat of the Stanley family.
[1] Lord James was actively involved in the conflict between the Murrays and the Frasers of Beaufort over the Lovat succession.
In 1697, after his sister Lady Amelia Murray escaped from her forced marriage to Simon Fraser (later 11th Lord Lovat), he and his brother Lord Mungo Murray commanded the troops sent to harry Fraser lands.
After Simon Fraser was declared an outlaw in September 1698, the Murray brothers led around 600 Athollmen and Lowland soldiers into Stratherrick with a view to capturing him.
They were ambushed by the Frasers and forced to surrender after their retreat to Inverness was cut off at Alt nan Gobhar, the Blacksmith's Burn.