Located in New York City on the university's Morningside Heights campus, its collections span more than 4,000 years, from early Mesopotamia to the present day, and span a variety of formats: cuneiform tablets, papyri, and ostraca, medieval and Renaissance manuscripts, early printed books, works of art, posters, photographs, realia (such as mathematical instruments and theater models), sound and moving image recordings, and born-digital archives.

The college exhibited an interest in acquiring significant books early in its history during the presidency of William A. Duer (in office 1829–1842) when it subscribed to the elephant folio edition of John James Audubon's The Birds of America, published from 1827 to 1838.

The 1881 bequest of Stephen Whitney Phoenix, the well-travelled scion of a New York merchant family, noted genealogist, and college alumnus, brought Columbia its first collector's library, around seven thousand rare editions and manuscripts.

Particular highlights of the Phoenix gift include a 15th-century French Book of Hours, a Jean Grolier bound Aldine edition of philosopher Iamblichus’ works, Shakespeare’s First Folio, and original drawings by inventor Robert Fulton.

Gottheil arranged the gift by Temple Emanu-el of New York City of its distinguished library of 2,500 printed books and fifty manuscripts of Hebraica in 1892, which placed Columbia's Judaica collection among the top in the country.

Four years after the Temple Emanu-El gift, in 1896, Columbia President Seth Low (in office 1890–1901), decided to make the college a university and to further expand the library so that it could support graduate level research.

The beginning of the active acquisition of collections of original manuscripts, autograph letters and documents was marked by Trustee William Schermerhorn's gift in 1902 of New York Governor De Witt Clinton’s papers.

Assembled by alumnus and historian of religion Acton Griscom and donated by him in 1920, it consists of several thousand books and manuscripts concerned with the heroine of French history.

Just before the official formation of the department, textile industry magnate and ardent opponent of the metric system, Samuel S. Dale donated his library on weights and measures, which was accepted by trustees on June 3, 1930.

In 1941, the seed of the graphic arts collection was planted when Columbia bought the library of the American Type Founders Company, built by Henry Lewis Bullen.

Over time, the biographical collection grew to include interviews with notable figures in philanthropy, business, radio, publishing, filmmaking, medicine, science, public health, law, military, architecture, and the arts.

As the library's collecting scope began to encompass non-Western peoples and subjects in the 1950s, it also slowly expanded to include women and racial and ethnic minorities.

And, in 1955, Frances Perkins, Secretary of Labor under President Franklin Roosevelt, the first woman to serve in a cabinet position, and Columbia alumna, donated her papers to the university.

The following year, the library acquired the papers of John Jay, former New York governor, U.S. Secretary of State and Foreign Affairs, and the first chief justice of the U.S. Supreme Court (1789–1795).

Further amplifying the library's literary holdings was Solton Engel, an attorney and alumnus who donated more than five hundred rare items, including Shakespeare's third (1663) and fourth (1685) folios.

Much of this expansive growth occurred under the leadership of director Kenneth A. Lohf, who, between 1967 and 1993, saw the rare book collection increase in size by 275,000 volumes and the addition of 21 million manuscripts.

In the 1990s and 2000s the Rare Book & Manuscript Library began to expand its collections relating to African American history and culture, placing a particular focus on Harlem.

James, poet Amiri Baraka, Dance Theater of Harlem founder Arthur Mitchell, and former New York City Mayor David Dinkins.

As early as 1959, Columbia University historian Richard B. Morris led an effort to publish a four volume set of all previously unpublished writings by John Jay.

In 1997, with the appointment of a new editorial advisory board, the library decided to digitize the source material gathered by the Morris team, and to commence a new print series comprising seven volumes.

The web archive is an initiative to systematically capture and preserve human rights websites to enable ongoing access to information that may be ephemeral and at-risk of disappearing.

In addition to capturing the web-based information generated by organizations whose print records the center holds, the web archive includes hundreds of other organizational and individual websites.

The origins of University Archives can be traced to the Columbiana collection, a vast store of Columbia memorabilia including documents, records, artifacts, photographs, and books that was created in the late 19th century and endowed as a department in 1930.