The noble Rantzau family owned this estate before it was sold to Frederick I, Landgrave of Hesse and King of Sweden, in 1739.

When he died in 1808 without progeny, the issue of King Frederick went extinct, and Panker was placed into a trust in accordance with the Prince's testament.

Subsequent trustees included Charles' son and heir Prince Frederik of Hesse, who died in Panker in 1845.

Upon Frederik's death the Panker inheritance passed to the Rumpenheim lineage of the House of Hesse, half-Danish royalty.

Nowadays it belongs to the Hessian House Foundation and serves as residence of some family members of the Rumpenheim line of landgraves of Hesse.