In 1525 during the German Peasants' War, Conrad was among the defenders of the Fortress Marienberg at Würzburg which was under siege.

Also despite the Emperor's admonitions, he gave excuses and refused to personally attend meetings of the Imperial Diet.

In August 1544 after reigning only four years he died which apparently solved the problem of his never having completed his ordination vows or formal installation as bishop.

Just before dying, Conrad gave Wilhelm von Grumbach 10,000 gold florins as a gift, without obtaining the consent of the Cathedral Chapter.

When, after the death of Conrad, Melchior Zobel of Giebelstadt, the new Prince Bishop asked for the money back from Grumbach.

Kretzer was captured on the French border in Schaumburg Castle in what was then Lorraine, confessed to the act in 1558, but hanged himself before he could be tried.