He repeatedly came up against resistance from knights who had refused to recognize their vassal (tenant) status, and 21 of them - the lords of Eltz, Waldeck, Schoneck, and Ehrenberg - agreed on a formal defensive alliance.

That Baldeneltz was probably built in a very short time to put the lords of Eltz under pressure, can be seen inter alia from the building material used: mostly small pieces of broken stone from the vicinity of the siege castle itself.

[1] It was held together with a strong clay mortar, which however cannot withstand a Central European climate in the long run.

Baldwin now displayed his negotiation skills and magnanimity in drawing his opponent onto his side: after Trutzeltz had been effectively "legitimized" by the peace treaty with the other knights and its survival secured by them, he transferred it in 1337 to John of von Eltz as a fief and made him a hereditary burgrave.

[4] On 9 January 1354 King Charles IV enfeoffed Archbishop Baldwin of Trier with Eltz Castle for his loyal service against Emperor Henry VII.