Holbæk is first mentioned in a document from 1199 in which Bishop Absalon presents in to Sorø Abbey.

The castle was held in fee by royal vassals and Holbæk Ladegård served as the administrative centre of the land.

Holbæk Ladegård with its tenant farms was instead sold to Henrik Thott, the owner of Boltinggaard.

He had no male heirs and the two estates were therefore entailed to his nephew, Hans Diderich de Brinck-Seidelin.

In 1769, Brinck-Seidelin established a stamhus (family trust) from the estates Holbæk Ladegård, Hagestedgård and Eriksholm (bought 1758).

[1] Holbæk Ladegård was then sold to Hans Peder Kofoed, a merchant and shipowner from Copenhagen.

The land was then sold off in lots for new housing while the buildings and park were turned into a folk high school.