He was the eldest son of wealthy land owner Leroy Stafford and his second wife, Elizabeth Calliham.

[1][2] The Staffords had ten children—George Waters, Sally, Elizabeth Eloise, David Theophilus, Cornelia Knox, Julia Laura, Jesse Wright, Rosalind, Leroy Augustus Jr., and Kate Keary.

He enlisted in the "Rapides Volunteers" (Company E, 3d Louisiana Infantry Regiment)[1] and later served in Captain Ben McCullough's Texas Rangers.

Stafford rose through the ranks and became the regiment's colonel in early 1862 when Richard Taylor was promoted.

Gen. William E. Starke early in the Battle of Antietam, Stafford was elevated temporarily to command of the brigade.

The Gen. Leroy A. Stafford Camp #358 of the Sons of Confederate Veterans in Alexandria, Louisiana, is named in honor of the fallen general.