It grew out of the National Conservatory of Music （Guoli yinyueyuan 国立音乐院）, which was established by Cai Yuanpei on November 27, 1927.

A six-year secondary school and a three-year elementary section were established in 1953 and 1956 to prepare better candidates for tertiary education, thus forming a self-contained system with a complete curriculum of music and academic education.

The conservatory has established six performing groups: Symphony Orchestra of Shanghai Conservatory of Music, New Ensemble, String Quartet, Percussion Ensemble, National Music Orchestra and Choir.

Shanghai Conservatory of Music maintains close relationships with many first-class conservatories and famous musicians, including collaborations with schools in the US, France, UK, Russia, Netherlands, Australia, Austria, Germany and Japan.

Many internationally well-known musicians such as Isaac Stern, Itzhak Perlman, Yuri Shishkin,[4] Leon Fleisher, Pinchas Zukerman, Seiji Ozawa, Simon Rattle, Mstislav Rostropovich, and Yo-Yo Ma have served as honorary or guest professors.