Five acres of ground were purchased from Herman Bruening for burial purposes by the St. Johannes Cemetery Association.

The Articles of Incorporation for the cemetery association were filed for the Dubuque Evangelical Lutheran St. John's Church at the Dubuque County Courthouse on September 17, 1864.

The signers of the articles were Paul Bredow, Daniel Schmalz, Adam Schnellbacher, John Funk, Frederick Wehland, and Andrew Heinzman.

It is situated on Davis Avenue (the same street that Mount Calvary Cemetery is located).

Throughout the years the cemetery has served primarily all of the Lutheran congregations of the area.