Savage served as Master-General of the Ordnance and Constable of the Tower, and was briefly commander-in-chief of the forces in lieu of James Butler, 2nd Duke of Ormonde until his death in 1712.

He served abroad in 1702 under Marlborough, who formed a high opinion of his military capacity and who recommended him for the command of a force for an invasion of France in 1706.

The expedition was eventually diverted to Portugal, and Rivers, finding himself superseded before anything was accomplished, returned to England, where Marlborough procured for him a command in the cavalry.

Rivers now met with marked favour at court, being entrusted with a delicate mission to the Elector of Hanover in 1710, which was followed by his appointment in 1711 as Master-General of the Ordnance, a post hitherto held by Marlborough himself.

Richard Savage, the poet, claimed identity with Lady Macclesfield's son by Lord Rivers, but though his story was accepted by Dr Johnson and was generally believed, the evidence in its support is faulty in several respects.