The building's symbolic importance was such that, after the military coup that led to Perón's downfall in 1955, the dictators who subsequently took power ordered its complete demolition, to erase all traces of its former occupants.

The history of Unzué Palace is documented from the end of the Juan Manuel de Rosas government (1835-1852), when three Englishmen purchased land for the construction of a summer home.

Signed plans from the early 1900s confirm that the family hired the architects Martín Vismara and A. Gonsalez to manage building renovation work.

In 1910, during the Centennial International Exposition, Unzué was used as the main site for the Exhibition of Health & Hygiene, which showcased several technical advances of the time including the flushing toilet, new sewer systems, and medical instruments.

The Great Depression of 1930 affected the Argentine agricultural export model significantly, and those families who had benefited now lost both income and power, which led to many being forced to sell their residences.

In 1938, the president Roberto Marcelino Ortiz decided to found a Garden of Childhood (Jardín de Infancia), moving the school faculty to the site.

It wasn't used extensively by presidents however, the first to do so being Edelmiro Farrell, who occasionally resided there on weekends away from the main presidential palace located in downtown Buenos Aires.

[3] Decree 14.576,signed by Aramburu, stated that: ...The operating costs and necessary repairs to be carried out on the aforementioned property exceed the sum of one million seven hundred thousand pesos in national currency, an investment that would not be proportional to the intrinsic value of the building; while it is true that in previous times it has served as the Presidential Residence, it is no longer possible to carry out new renovations or adapt it from a functional and architectural standpoint to meet the minimum characteristics that a property intended for the accommodation of the President of the Republic must offer....[3]Aramburu was the first ruling Argentine leader to use the Quinta de Olivos as a presidential residence, which it continues to be to the present day.

In the library's lobby, several bronze plaques recognize both Eva Perón and the demolition of Unzué Palace, as an attempt to erase a part of Argentina's history.

Occupying a significant site above a steep bank or barranca, with elevated gardens that descended northward, the building overlooked the current Avenida del Libertador and bordered Calle Agüero, where the ancillary functions were located.

Two rooms were joined to create a library, whose walls were lined with light walnut wood panelling and shelving, and a included a veined marble fireplace.

It contained a wide variety of plant species, including magnolia, cedar, araucaria, palm, chestnut, orange and fig trees, in addition to a lake and a planned aviary.