The palace was built in the second half of the 17th century for the noble and wealthy Pesaro family, a project by the Venetian architect, Baldassarre Longhena, who also designed the church of the Salute and Ca' Rezzonico.

Longhena was inspired by Sansovinian classicism when designing Ca’ Pesaro, creating expressions of a new and sumptuous harmony, with its complex and powerful composition, yet well-balanced.

In 1902, thanks to the bequest of the Duchess, the city council decided to use the palace to host the Modern Art municipal collection, which had been started in 1897, when the second Venice Biennale was held.

Shortly afterwards, between 1908 and 1924, it also was used to host the Bevilacqua La Masa exhibitions, which, in lively contrast with the Venice Biennale, favored a generation of young artists, including Boccioni, Casorati, Gino Rossi, Juti Ravenna and Arturo Martini.

This collection of oriental objects was brought back from a stay in Asia by Prince Henry of Bourbon-Parma, Count of Bardi, at the end of the 19th century, and bequeathed to the Italian state.