The son of George Douglas, 1st Earl of Angus and Princess Mary of Scotland, he was a grandson of King Robert III.
During the king's captivity, Scotland was ruled by his uncle Robert Stewart, Duke of Albany, who had been in no hurry to pay his nephew's ransom.
Whatever the machinations that followed, Angus was not included on the final list of hostages, but was one of the party of Scots nobles who met their King at Durham, in 1424.
The castellan, George II, Earl of March, had previously been made a ward of the King, and the garrison surrendered the castle bloodlessly.
This help materialised in the spring of 1435 when Sir Robert Ogle, the Governor of Berwick upon Tweed, with Henry Percy and 4000 men marched north to retake the Castle.
Angus, with Hepburn and Alexander Ramsay of Dalhousie, decided not to undergo a siege and engaged with the English forces at the Battle of Piperdean, near to Cockburnspath.
Angus continued to consolidate his estates, often at the expense of his cousins the Black Douglases, taking positions and fortresses previously held by the Earls of Douglas, such as Lintalee and finally Hermitage Castle for a time.