The name comes from the church of John the Apostle, (Russian: Иоанн Богослов, Ioann Bogoslov), which from the early 18th century had a burial ground attached, mostly for those who died in the nearby military hospital.
In 1915–1916, a new wooden three-fronted church of John the Apostle was built at the cemetery to the design of architect Viktor Bobrov [ru].
In Soviet times the cemetery became the burial place of many prominent scientists, cultural figures, and military officials.
These are found in the northern part of the cemetery, marked by a hill topped with a small obelisk.
In October 2000, a newly rebuilt wooden church of John the Apostle was consecrated.