At the age of 8, Pickett moved with his father, William R. Pickett, from North Carolina to the frontier of Autauga County, ceded to the United States four years earlier by the Creek Indians in the Treaty of Fort Jackson.

It was part of the federal government's effort to extinguish land claims and gain Indian Removal from the Southeast, in order to enable settlement by European Americans.

William Pickett built a home near Autaugaville, and a mill building and trader's post on Swift Creek.

While growing up, Albert befriended many of the Creek and frontier traders who frequented his father's store.

His History of Alabama and Incidentally of Georgia and Mississippi from the Earliest Period was published in 1851, with the copyright renewed in 1878 by Mrs. Sarah S. Pickett.