His maternal grandparents were Evelyn Pierrepont, 1st Duke of Kingston-upon-Hull and his first wife Lady Mary Feilding.

With the death of his elder brother in 1746, he became known by the courtesy title of Viscount Trentham until he succeeded his father as Earl Gower in 1754.

Between 1775 and 1778, Stafford proceeded to make substantial alterations to his home at Trentham Hall based on the designs by Henry Holland.

By 1779, Gower resigned from the cabinet being frustrated by what he saw as the North administration's inept handling of the American Revolutionary War.

Instead, he became a key figure in bringing about the fall of the Fox-North coalition, and was rewarded with the position of Lord President once again in the new administration of William Pitt the Younger.

Although he soon exchanged this office for that of Lord Privy Seal, and gradually began to withdraw from public affairs, he remained a cabinet minister until his retirement later in 1794.

In 1799 he (or his immediate family benefit trust) was estimated the fifth-wealthiest small family unit in Britain, owning £2.1 million (equivalent to £260 million in 2023)), having assets in land, mining and arterial canal-toll rights having speculatively invested in the latter projects, much of which was in Staffordshire's Black Country.