Work on the 210-acre (85 ha) Mount Olivet Cemetery site was announced in November 1895; it was planned to be a non-sectarian cemetery on the western slopes of San Bruno Mountain which would be subdivided into sections reserved for fraternal organizations such as the Native Sons of the Golden West, Knights of Pythias, Improved Order of Red Men, and Ancient Order of Foresters.

The site adjoins the older Hills of Eternity and Home of Peace Jewish cemeteries, separated by Hillside Boulevard, which was then known as San Bruno Avenue.

[2] By that time, the debate on keeping cemeteries within San Francisco had begun to trend toward relocating the dead, and the development of Mount Olivet and Cypress Lawn in Colma was given as evidence that community "will probably be made the receptacle for all the dead of [San Francisco] in the very near future.

[5] The oldest buildings onsite include the stone chapel (1896) and columbarium (1915), both designed by William H. Crim Jr.[6]: 87 There are two large memorials at Olivet: one dedicated to the Sailors Union of the Pacific by Governor Earl Warren in 1946 in memory of the 6,000 United States Merchant Marine sailors who died in World War II,[7] and another named "Showman's Rest", erected by the Showfolks of America in 1945.

[6]: 91–92 [8] By that time, when Robert Royston was engaged to perform landscape architecture for the site, the name had been changed to Olivet Memorial Park.