The present manor was built in 1825, following a design by Karol Podczaszyński in the neoclassical style, by the order of the Governor General of Lithuania, Alexander Rimsky-Korsakov.

In the area on the right bank of the Neris River opposite the St. Peter and St. Paul's Church in Antakalnis, the Royal Manor, so called Derevnictva (Polish: Derewnictwo), was established in the middle 16th century by the King Sigismund Augustus in order to supply vegetables and meat to Vilnius Castles.

In the mid-19th century the main palace was transformed into a guesthouse by prominent doctor and public activist Julian Titius that became a cultural center in Vilnius, often visited by Stanisław Moniuszko and Józef Ignacy Kraszewski.

[5] The area had been used to hide the bodies of Lithuanian residents – mostly resistance fighters against the Soviet occupation, Nazi collaborators and Armia Krajowa soldiers – who had been executed between 1944 and 1947 by the NKGB and MGB in the Vilnius' KGB Palace prison near Lukiškės Square (now Museum of Genocide Victims).

The area surrounding the Tuskulėnai Manor was referred to as Tuskulionys (Russian: Tuskuljany;[6] Polish: Tuskulanum[7]) until World War II.

This area was also known by the colloquial placename Losiovka, named after A. Losev, colonel of Special Corps of Gendarmes and later general of the Russian Empire, who owned the folwark of Tuskulėnai in 1869.