The cemetery originated in 1836, five years after Princeton's founding, and was originally owned by Reverend Lucien Farnham as a burial ground for members of the Hampshire Colony Congregational Church.

The city of Princeton leased the burial ground from the church in 1862 and bought a large plot of adjacent land, and by the next year they had combined the two plots and named them Oakland Cemetery after the oak trees there.

The cemetery included a public picnic area, carriage paths, and a Gothic Revival office building.

Many of the graves in the cemetery feature large, artistic monuments, including obelisks made from white bronze, statues of angels or the deceased, stone tree trunks, and Art Deco box tombs.

The largest monument in the cemetery marks the grave of pioneer and local historian Nehemiah Matson; it is 34 feet (10 m) tall and made of 44 tons of granite, and it cost $4,600 when it was built in 1884.