The house is located at the site of a former boom gate across the old road from Lyngby Kongevej to Frederiksdal and belonged to the gatekeeper (vangemanden).

The village of Virum's pastures began at the site and the gate was to keep cattle from passing onto the main road.

It was the residence of Frederick, Hereditary Prince of Denmark and Princess Sophie Frederikke.

The bridge was renewed when Sorgenfri station was established in connection into an S-train line in the early 1930s.

For many years, Lottenborg belonged to the restaurateur Ernst Høegh-Hansen and then his widow Agnete who died in 1993.