Jackson was one of fifteen children born in Clay County, Kentucky, as the daughter of Oliver Perry Garland and Deborah Robinson.
However, due to the extensive subdivision of land in southeastern Kentucky over the course of the latter half of the nineteenth century, her father's profession grew to be inviable.
The family moved to East Bernstadt, Kentucky in Laurel County where Oliver opened a general store selling groceries to miners on credit and became a pastor at the Missionary Baptist Church in town.
[3][5] In December 1931, Jackson traveled to New York City to support and raise money for striking Harlan coal miners,[6] at one point appearing before an estimated crowd of 21,000 at the Bronx Coliseum.
In the mid-1930s, she performed in New York City together with Woody Guthrie, Pete Seeger, Earl Robinson, Will Geer, her half-brother Jim Garland, and her half-sister Sarah Ogan Gunning.
Folklorist Archie Green became very frustrated during interviews with her, due to her "elastic responses", inconsistent elaborations and "flexible dates."