As a minister of state, his accomplishments included the modernization of higher education and the suppression of ecclesiastical influence.
Though replaced by Joseph Sonnenfels under Leopold VII (1790-1792), he was again in charge of suppressing opposition under Francis II (1792-1835).
The Von Pergen family originated from the Habsburg Netherlands and migrated to Austria in the sixteenth century.
He married Philippine Gabriele, a daughter of the president of the Imperial Chamber Court, Philipp Karl von Groschlag, and former lover of the future emperor, Joseph II when living in Frankfurt am Main, before moving to Vienna.
Von Pergen himself led a lavish lifestyle and was a patron of the arts, and in 1782 acquired the Schloss Pottenbrunn whose park he embellished with romantic buildings.
He was president of the Imperial occupation administration in his capacity as minister plenipotentiary of the Lower Rhenish-Westphalian Circle.
Pergen had a good knowledge of the estates of the realm from his time in Mainz and other embassies, and warned against a single policy for Austria.
[4] In this he argued that the Emperor (Joseph) should demonstrate Liebe zu Teutschland, and advised him to "defend the System with all [...] possible strength."
This was anti-clerical and represented the reforms of the Enlightenment, which proved difficult to implement in the face of resistance from Maria Theresa and conservative-minded circles in the court.
Pergen wanted to unify the system of internal security in the hereditary lands in a centralized and modernized form.
His goal of a central, independent police administration was realised by 1789, by which time he was a Minister of State with an appropriate department to administer.
In 1801, he was also responsible for censorship and from then on Francis II and his successors would receive the latest police reports every morning.
Due to the fear of a spillover of the French Revolution to Austria, he advocated monitoring the population even more meticulously.
Pergen laid the foundations by his organization, for Josef von Sedlnitzky during the era of Metternich from 1814 to 1848, to pursue his antiliberal policies.