After the Reformation in 1536, the estate was confiscated by the Crown but the fief was as a sort of pension granted to the last Catholic bishop, Ove Bille.

[1] Ove Billes was succeeded by Peder Christensen Dyre but lost his fief when he was found guilty of perjury in connection with a legal dispute in 1558.

The crown then ceded the ownership of Yølløsegaard to Peder Oxe in exchange for other property.

Peder Oxe was in 1552 made a pricy counsellor but had to resign from all his public offices after a controversy with Christian II in 1558.

His estates in Denmark was then ceded to his sworn enemy, Herluf Trolle, the founder of Herlufsholm Abbey School.

Peder Oxe returned to Denmark during the Northern Seven Years' War and got his estates Tølløse, Gisselfeld and Løgismose back.

Their daughter, Margrethe Friis, in 1665 ceded the estate to Christian Skeel in exchange for other property.

On his death, Tølløsegaard passed to his son-in-law, Gustav Frederik Ysenburg-Büdingen, who supposedly came to detest the premises after his young daughter died in an accident on the estate.