The current main building is from 1915-16 but its design is inspired by traditional Renaissance style manor houses.

Broksø (formerly Broxøe) was established in 1650 when Frans Lykke til Gisselfeld merged two farms, naming the estate after his wife Elisabeth Brok.

The suffix -ø (meaning "island") is a reference to the main building's location on Spragelse ø, a small peninsula surrounded by bogland.

[1] In 1601, Lykke was sentenced to a great fine after accusing Queen Sophie Amalie of adultery in a letter which was exposed to the king.

When he fled to Skåne instead of paying, he was sentenced to death for Lèse-majesté, was executed in his absence and had his property in Denmark confiscated.

Otto Krabbe's widow, Birgitte Skeel, kept Holmegaard and Broksø after her husband's death in 1719.

The front of the building features a three-storey median risalit with a monumental staircase and two lower corner pavilions with hip roofs.

The facade towards the gardens feature two projecting, octagonal corner towers with conical red tile roofs.