In the palace park are found, among other things, a "temple of Diana" (originally dedicated to Neptune)[1] and the burial site of Pompe, the dog of King Charles XII.

[4] Three medieval villages at the location — Ösby, Bolstomta, and Lundby — were bought by Lord High Admiral Carl Carlsson Gyllenhielm (1574–1650) in the 1620s and subsequently unified into a single estate named "Karlberg" after himself.

[7] Following the Reduction, De la Gardie lost his influence and most of his fortune, and since King Charles XI declined to buy the palace, De la Gardie was finally forced to hand it over to Johan Gabriel Stenbock to settle a debt, and in 1683 Stenbock took over the newly rebuilt palace, only to sell it to the king in 1688.

[6] While the Three Crowns Castle was being rebuilt in the early 1690s,[6] and following the disastrous fire which destroyed it in 1697, the royal family chose Karlberg as their temporary home.

[5] The red panelled log houses west of the main building are believed to date back to the 1720s, while the stables carrying four sandstone vessels were designed by Carl Hårleman (1700–1753) under Frederick I (1676–1751) in the 1730s, thereafter rebuilt into barracks in the 1790s.

Furthermore, in 2001 an archaeological examination of a nearby burial site, associated with one of the villages out of which Karlberg once was created, unveiled fragments of runestones — including one from an image stone and another featuring the proto-Norse Elder Futhark.