[1] The church was originally conceived as a house of worship for the Johnstone family on their Annandale Plantation, now destroyed.

[2][3] Almost no primary records for the construction of the church survive, but it is commonly believed by architectural scholars that English-born architect Frank Wills designed the Chapel of the Cross for Margaret Johnstone.

Johnstone had much of the work on the church performed by people who were enslaved by her and imprisoned on her plantation Slavery in the United States; they made all of the bricks by hand on-site.

In 1979, a few years after the addition of the church to the National Register of Historic Places, the United States Department of the Interior awarded a $50,000 grant to finish the restoration.

The first is that of Annie Devlin, a former governess who died at the Annandale mansion in June 1860 and was purported to haunt it until the night it burned in 1924.

Helen, the youngest daughter of John and Margaret, and Henry, descended from the founder of Vicksburg, met at her sister's home, Ingleside, in December 1855.

[4] The same folklore is repeated in Norman and Scott's Historic Haunted America and again in Jan Warner's booklet Shadows of a Chapel.