In 1787 Charteris' uncle Lord Elcho (who but for his attainder would have been 6th Earl of Wemyss) died.
At the time eldest sons of Scottish peers were not allowed to represent Scottish constituencies in Parliament, and after a debate on the matter Charteris had to vacate his seat.
Although it was later established that the Earldom of Wemyss remained forfeit and his father was not after all a Scottish peer, Charteris did not attempt to re-enter Parliament.
Charteris died on 20 January 1808 at Amisfield House, East Lothian, and was interred at St Mary's Collegiate Church, Haddington.
They had one son and four daughters: In 1818—after Francis Chateris's death—his widow Susan Chateris (by then the Dowager Lady Elcho) changed her surname to Tracy as a condition of inheriting her uncle Robert Tracy's estate, on the death of her elder sister.