He assumed in 1870 (alongside his father) by Royal licence the additional surname of Sackville.
He remained out of the House of Commons for the next twenty years trying unsuccessfully to return to parliament in the 1894 Wisbech by-election.
Stopford-Sackville married Edith Frances, daughter of William Rashleigh, in 1875.
Sackville-Stopford survived her by over twenty years and died on 6 October 1926, aged 86.
A reredos was installed in his memory in St Peter's Church, Lowick.