In 1574, the responsibility of his fiefs were taken over by his brother King John III, who managed them as his guardian because of his mental condition.

Magnus died at the Manorhouse of Kungsbro, outside Linköping, in 1595 and is buried in the Bridgettine Abbey Church in Vadstena.

There are many stories about the "Mad Duke", for instance about how he saw a mermaid in the castle moat and threw himself out of a window, nearly drowning himself.

There is no proof that this event ever took place, but the story might have evolved from an incident in 1563 when he fell in the moat, during the construction of the castle drawbridge.

Geijer and Afzelius note that the alleged mermaid incident appears to have found its way into the ballad "Hertig Magnus och Hafsfrun" ("Herr Mannelig").