The cemetery was designed in 1870; according to the plans of the Frankfurt landscape architects Karl Jonas Mylius and Alfred Friedrich Bluntschli who were awarded for their project per angusta ad augusta (from dire to sublime).

[2] This argument became even more aggressive when the city announced that it did not want an official Catholic opening of the new cemetery – but gave a substantial amount of money toward the construction of a segregated Jewish section.

In the end, the groups reached an agreement resulting in the Catholic representatives opening the Central Cemetery with a small ceremony.

Due to refraining from having a large public showing, the new cemetery was inaugurated almost unnoticed in the early morning of 31 October 1874 by Vienna Mayor Baron Cajetan von Felder and Cardinal Joseph Othmar Rauscher to avoid an escalation of the public controversy.

[5] Opposite the cemetery's main gate, across Simmeringer Hauptstrasse, is the Feuerhalle Simmering, Vienna's first crematorium, which was built by Clemens Holzmeister in 1922 in the style of an oriental fortress.

It used to be called Dr. Karl-Lueger-Gedächtniskirche (Karl Lueger Memorial Church) because of the crypt of the former mayor of Vienna below the high altar.

The burial vault is located beneath the sarcophagus, with stairs leading down to a circular room whose walls are lined with niches where urns or coffins can be interred.

On 1 November 2023, unidentified vandals set a fire and sprayed swastikas on external walls overnight in the cemetery's Jewish section.

The dead are buried according to Austrian law, in a coffin, in contrast to the Islamic ritual practice: burial in a shroud.

An area of the Central Cemetery has been set aside for this purpose centered around a stupa, and was consecrated by a Tibetan monk.

[17] The Church of Jesus Christ of Latter-day Saints in Austria celebrated the dedication of a hectare-sized plot set apart for the Mormon deceased in the Vienna Central Cemetery, on 19 September 2009.

The "Zentralfriedhof" stop on the Vienna S-Bahn (metro suburban railway) is close to the old Jewish part of the cemetery.

The closest underground stop is "Simmering" (Vienna U-Bahn, line U3), about 2 kilometres (1.2 mi) from the cemetery.