Its origin was in St. Petersburg in the Rumyantsev house or mansion, building number 44 on the English Embankment overlooking river Neva.

Maintenance difficulties were among the reasons for the shift of Rumyantsev Museum to Moscow, despite it being affiliated to the Public Library in Saint Petersburg since 1845.

A number of notable people used the library such as Dmitri Mendeleev, Konstantin Tsiolkovsky, Fyodor Dostoevsky and Leo Tolstoy.

[4] Sergei knew that his brother Nikolay had wanted his personal art and book collection to be accessible to society.

[4] For this he decided to use the Rumyantsev house or mansion [ru],[4] located on the English Embankment in St. Petersburg,[5][6] where Nikolay's collections were already stored.

[20] Assistant's included Friedrich von Adelung who was known for collecting foreign reports on Russia.

[22] The sculpting on the pediment is inspired by mythology connected with Mount Parnassus, Apollo-Musagets, Mnemosyne and the Muses, a shout-out to the buildings' purpose as a museum as well as a reference to Rumyantsev.

[23] At a later date the pediment was engraved with the words of Nikolay, "for (the) good (of) enlightenment",[b] also translated as "for the benefit of education",[27] with the entire engraving reading as, "From the State Chancellor Count Rumyantsev for the good enlightenment" (Russian: ОТb ГОСУДАРСТВЕННОГО КАНЦЛЕРА ГРАФА РУМЯНЦЕВА НА БЛАГОЕ ПРОСВЕЩЕ).

[34][17] In May 1861 under the vision and advice of Nikolai Vasilyevich Isakov [ru] and his predecessor a decree for the "first public museum in Moscow" was framed and during the following year approved by Alexander II of Russia, then the emperor.

[38][8] Katia Dianina of the department of Slavic Languages and Literatures of the University of Virginia writes that the shift of the museum to Moscow and its opening was the beginning of the city's "cultural renaissance".

[39] The museum has undergone a number of changes in name,[12][40][41] Its collection included paintings from the Old Masters and some more recent artists.

[47] Pavel Tretyakov gave the museum Vladimir Borovikovsky's "Portrait of Amvrosy Podobedov".

[19] In 1915 a new gallery opened with different floors holding Italian, French, Dutch and Russian works.

[55][56] Konstantin Tsiolkovsky, without the means to enroll in formal education during the period of 1873–1876 in Moscow, used the library's scientific literature during those three years.

[61][12] This change in name and designation also allowed for the highlighting of the working conditions in the library, including the health of the staff, one-fourth who had tuberculosis.