The cemetery was established in 1873, and includes a grouping of headstones marking the remains of 58 black soldiers and sailors who served in the American Civil War, and a monument honoring these veterans stands over their graves.

Other notable elements include the Potter's Field, O’Rourke Mausoleum, and the West Point Cemetery entry sign.

The Civil War soldier depicted on the top of the West Point Monument is Norfolk native Sergeant William H. Carney of the 54th Massachusetts Volunteer Regiment.

James Fuller (1846–1909) was the primary force behind the monument's installation.

Fuller was a former slave and a Civil War Veteran quartermaster in the First United States Colored Cavalry, and was later elected as the first black Norfolk City Councilman.