It is likely that a wooden mansion of Mikołaj "the Black" Radziwiłł stood in the same site in the 16th century.

[1] The current building was constructed by the order of Janusz Radziwiłł from 1635 to 1653 according to the design of Jonas Ullrich.

Eventually, it was restored in the 1980s and a division of the Lithuanian Art Museum is located there today.

Currently, this branch of the Lithuanian National Art Museum contains Old Master paintings by Ludovico Lipparini, Giuseppe Rossi, Antonio Bellucci, Carlo Dolci, Giovanni Paolo Panini and Bartholomeus Spranger.The Palace currently is under the restoration process during which lost parts of the building will also be rebuilt.

The project is expected to be completed in ~2023, commemorating the 700 years jubilee of the first mention of Vilnius in 1323.