Little is known about his early life, but a story that does survive attests to his father's fecundity and possibly also to Robert's own spirited approach.

[3] Duff received the command of the 44-gun Anglesea on 23 October, and remained in her off the Irish and English coasts until the signing of the Treaty of Aix-la-Chapelle in 1748 that ended the War of the Austrian Succession.

[3] While at anchor in Quiberon Bay on 20 November, his scouts reported a large French fleet to the south of Belle Île.

Duff saw action at the capture of Martinique during January and February 1762, but owing to his seniority, he refused to serve as Rodney's flag-captain, and was sent home.

He was part of Horatio Nelson's fleet, and was assigned to command the inshore squadron watching the enemy in Cadiz prior to the Battle of Trafalgar.