Leopold II succeeded as margrave upon his father's death in June 1075, at the time when the Investiture Dispute broke out between King Henry IV and Pope Gregory VII.

However, he switched sides under the influence of his wife Ida and Bishop Altmann of Passau, a loyal supporter of Pope Gregory who was expelled from his diocese by Henry's forces in 1078.

Altmann fled to Austria and Margrave Leopold had to face an invasion by royal troops the next year, which led to the final break with the king.

While Bishop Altmann of Passau stayed in Austria, the margraviate obtained a leading position in promoting the Gregorian Reforms, overruling the concept of proprietary churches and marriages of priests.

In 1089 Margrave Leopold helped pay for the construction of Melk Abbey in the Wachau region by donating extended premises high above the shore of the Danube for the new monastery.