Because in 1907 his wife, Antonina, delivered a healthy baby boy (Stanisław) after a worrisome pregnancy, Feliks commissioned a sculpture of the blessing Jesus for the front of the palace.

The palace gardens were designed by Édouard François André, French horticulturalist and landscape architect, and his son.

[4] In 1916, an attempt at palace expansion was started, with Heinrich Schwechten being requested to prepare designs for expanding the upper floors and veranda, but these projects were never completed.

Countess Antonina, her son, Stanislaw, his wife Wanda, and their daughter, Renia were forced to flee from their Palanga home during World War II.

[9] The museum holds the third largest amber specimen in Europe, the "Sun Stone", weighing over 3.5 kilograms, which has been stolen twice.

[10] The cultural and artistic exhibits of museum currently include a 15th-century ring, a 16th-century cross, and amber jewelry from the past four centuries,[9] as well as a number of modern decorations.