Stafford acquired portions of lands belonging to General Nathanael Greene through auction, and continued to assemble former Greene family lands so that by 1830 Stafford controlled 1,360 acres (550 ha) with 148 slaves.

In 1843 Stafford acquired 4,200 acres (1,700 ha) from P.M. Nightingale, a Greene descendant who retained Dungeness.

His heirs sold the property to Thomas M. Carnegie and his wife Lucy, who had also acquired Dungeness.

[3] The Stafford Mansion was built by Lucy Carnegie in 1901, for one of her children.

[4][5] The property is privately held under a life estate by a Carnegie descendant within Cumberland Island National Seashore.