Valdemar was the son of the Swedish princess Ingeborg Eriksdotter and Birger Jarl, from the House of Bjälbo.

Birger Jarl had been the de facto ruler of Sweden from 1248, before the reign of Valdemar, even under King Erik Eriksson.

In 1260, Valdemar married Sophia, the eldest daughter of King Eric IV of Denmark and Jutta of Saxony.

The following year, Jutta was placed in a convent and Valdemar was forced to make a pilgrimage to Rome to ask for the absolution of the Pope.

In 1288 Valdemar was imprisoned by King Magnus in Nyköping Castle (Nyköpingshus) and lived openly with mistresses, possibly new wives, in comfortable confinement.