The museum was founded in 1884 by Augustus Pitt Rivers, who donated his private collection to the University of Oxford with the condition that a permanent lecturer in anthropology must be appointed.

The display of many examples of a particular type of tool or artifact, showing historical and regional variations, is an unusual and distinct feature of this museum.

[5] This topological layout is based upon Pitt Rivers' theories; he intended for his collection to show progression in design and evolution in human culture from the simple to the complex.

Although this evolutionary approach to material culture is no longer appropriate in the modern display paradigm for archaeological and anthropological objects, the museum has broadly retained the original typological organization due to the Pitt Rivers Deed of Gift which stipulated that any changes to the displays "shall not affect the general principle originated by Augustus Henry Lane Fox Pitt Rivers".

It originally stood in front of the Star House in the village of Old Massett (Haida name Uttewas), on Graham Island, in British Columbia, Canada.

[16]: 41  By 1964, the university wanted to expand the science area onto the Pitts Rivers site and move the museum to a larger plot on Banbury Road.

[16]: 41  A round museum was proposed, covering a site larger than the Sheldonian Theatre, the Clarendon Building, and the Bodleian Library combined.

[16]: 41  In February 1966, the university greenlit the new museum and promised to keep the Banbury Road site free for three years while funding was arranged.

[16]: 43  In May 1968, Fagg suffered a stroke which left him in hospital until the end of the academic year and significantly limited his involvement in fundraising efforts.

Building work was completed in 2007, bringing the academic staff of the museum back to the site, and providing a laboratory for conservation of the specimens.

A new entrance platform was built to allow visitors to enter on the same level as the Oxford University Museum of Natural History and improves access for wheelchair users and parents with pushchairs.

[10] As part of this process the Pitt Rivers Museum is meeting with originating communities to address errors and gaps in the information it stores, and to discuss repatriation.