Recent research published in 2013 reveals that although Johnston never married, he was the father of four daughters by his manumitted mistress, Edith "Edy" Wood, of nearby Hertford, N.C.[3] Two of his girls died at the age of eight and nine in 1836, and his eldest daughter, Mary Virginia Wood Forten (daughter-in-law of wealthy African American abolitionist, James Forten), died in Philadelphia of tuberculosis in 1840, leaving behind her three-year-old daughter, the future diarist, poet, and equal rights activist Charlotte Forten Grimke.

[3] The girls continued to be raised with the Forten-Purvis clan while Annie Wood was adopted by Amy Matilda Cassey, daughter of African American Episcopal priest, Peter Williams, Jr. and wife of wealthy African American financier and benefactor, Joseph Cassey.

[3] James Cathcart Johnston paid for Annie Wood's education, made generous payments to her as she grew up, and promised her an "independence.

"[3][4] James C. Johnston died at the end of the Civil War and left Hayes to his "friend and advisor" Edward Wood of Greenfield Plantation.

His biracial daughter, Mariah was raised in the Wood home and was reared by her aunt Sarah Wood-Walker (house servant).

The main house, considered by architectural scholars to be "one of the South's most accomplished examples of a five-part palladian villa," was designed by English-born architect, William Nichols, Sr., famous for his early Neoclassical-style buildings in the American South and for designing statehouses for three Southern states.