Squire Peter Jonsson (later knighted) and his elder brother Sir Sune Jonsson, Lord of Flishult, Royal Councillor, the lawspeaker of Tiohärad (in inland Småland), together with their close relative Charles, Bishop of Linköping, were allies of the new king, Magnus IV of Sweden, in 1320 or 1321 and purchased dominus Efflerus, the bailiff of the deposed king, Birger of Sweden, from Viborg Castle.
There are toponymic indications of an influx of Southern-Swedish immigrants having settled in vicinities the of Viborg and on the coast west up to the Kymi River.
Peter and Sune recognized the new king and received important privileges, which effectively turned their holding of Viborg into an independent feudal fief, the start of a veritable margraviate.
Sune's son Erengisle, Earl of Orkney was a recorded owner of Kymmenegård Manor, in Viborg Province, the spot on which the town of Kotka later became erected.
They organised defences, guarded the area constantly, provided food and equipment, kept the fortress in shape, kept mercenaries and paid the military.
Apparently, the main reason for the chatelain of the Viborg Fief succeeding in keeping such an independent position, compared to other castles and their holders, was Viborg's extraordinary position as the easternmost outpost and the stronghold of the Swedish realm against the eastern neighbouring power from its attacks and desires to annex more land.
Important personages who held Viipuri as their fief were Bo Jonsson Grip, Krister Nilsson Vasa (1417–42), Charles Knutsson Bonde (1442–48, the future king), Erik Axelsson Tott (1457–81), Knut Posse (1495–97), Sten Sture the Elder (particularly 1497-99 when he was personally in residence, between his regencies), Eric Bielke and Count John of Hoya.
In 1534, King Gustav I of Sweden abolished the independent fief by deposing and exiling his brother-in-law John, Count of Hoya.
In 1320, lord Peter Jonsson (Haak) purchased the castle and its dominions from the bailiff Efflerus set there by the deposed king Birger.
Lord Nicholas Grabbe was the next commander of the Viipuri castle, 1534–45, but he did not receive the feudal privileges held by earlier chatelains.