It was erected as the Apostolic Vicariate of Nanjing in 1659 by Pope Alexander VII, and promoted to a diocese by Pope Alexander VIII on April 10, 1690.

Pope Pius XII elevated it on April 11, 1946 to the rank of a metropolitan archdiocese, with the suffragan sees of Haimen, Shanghai, Suzhou, and Xuzhou.

The archdiocese's motherchurch and thus seat of its archbishop is the Cathedral of the Immaculate Conception also known as the Shigu Road church.

There were a number of years without a bishop after the death of Paul Cardinal Yü Pin on August 16, 1978, because of the irregular relations between the Communist government and the Holy See.

On January 6, 2000, Francis Savio Lu Xinping was ordained the new bishop of Nanjing along with four others in an illicit ceremony in Beijing's South Cathedral.