It is located on the west side of the Rock River at 1827 North Washington Street.

Mount Olivet Cemetery was established in Janesville after the local cemetery located atop Courthouse Hill was moved to a new cemetery called Oak Hill at the northwest edge of the city.

Later, additional land was added to the cemetery.

A small Romanesque Revival style chapel was added to the cemetery in 1901.

It serves the parishes of St. William, St. John Vianney, St. Patrick, and Nativity of Mary in Janesville.