Frederik Georg Adeler inherited and lived on the large estate at the Gimsøy Abbey with a manor house and headquarters at Klosterøya in Skien.
After leaving the military, he was appointed county governor of Bratsberg on 30 July 1764, when he was 28 years old.
The investigations in connection with the Lofthus rebellion had brought to light a number of irregularities in his relationship, namely the illegal and unreasonable free rides he undertook on his many journeys between Kristiansand and Gimsøy in purely private matters.
In 1796 he married Caroline Rudolphine Schubart (1760—1798) who was the widow of the chamberlain Jacob Løvenskiold.
Adeler had only one surviving child, Anton Beatus, who was in poor health and who in 1818 married the widow Mogensine Bentzen.