From the Emperor Diocletian's tetrarchy (c. 300) they became the administrators of the four Praetorian prefectures, the government level above the (newly created) dioceses and (multiplied) provinces.
For some auxiliary troops, specific titles could even refer to their peoples: Roman provinces were usually ruled by high-ranking officials.
The most famous example is Pontius Pilate, who governed Judaea at a time when it was administered as an annex of Syria.
As Egypt was a special imperial domain, a rich and strategic granary, where the Emperor enjoyed an almost pharaonic position unlike any other province or diocese, its head was styled uniquely Praefectus Augustalis, indicating that he governed in the personal name of the emperor, the "Augustus".
After the mid-1st century, as a result of the Pax Romana, the governorship was gradually shifted from the military prefects to civilian fiscal officials called procurators, Egypt remaining the exception.