Major Bulloch moved his family from Savannah in 1838 to north Georgia to partner with Roswell King in establishing a cotton mill in the piedmont near the fall line.

Today, his plantation house known as Bulloch Hall has been restored and is listed on the National Register of Historic Places.

Bulloch also developed a plantation in the uplands, where his workers cultivated and processed short-staple cotton, the chief commodity crop.

Together, they had two sons: After Hettie died, Major Bulloch married on May 8, 1832, Martha "Patsy" Stewart (1799–1864), the second wife and widow of Senator Elliott.

Together, James and Patsy had four children, with the youngest two, Charles and Irvine, being born in Cobb County, Georgia after the family had moved from Savannah.