He married Elizabeth, a daughter of Margrave Leopold of Styria, and died childless on 12 October 1161, whereafter Herman succeeded him as Carinthian duke.
In December 1161 he was solemnly enthroned on the Duke's Chair in the Zollfeld plain, in the presence of Archbishop Eberhard of Salzburg and Patriarch Ulrich II of Aquileia.
With varying degrees of success he tried to consolidate his position in Carinthia by achieving the office of a Vogt protector over the Diocese of Gurk as well as over the Carinthian possessions of the Bishopric of Bamberg and the Patriarchate of Aquileia.
Upon the death of his uncle Margrave Engelbert III of Istria in 1173, Herman could not assume the margravial title, but inherited his personal possessions around the town of Sankt Veit which emerged as ducal residence.
About 1173, Herman married Agnes of Babenberg, daughter of Duke Henry II of Austria and widow of King Stephen III of Hungary.