Enlisting in the Confederate Army, he was named a captain in the 16th Georgia Volunteer Infantry before being promoted to lieutenant colonel.

During the Battle of Gettysburg, the 16th Georgia was among the troops that were poised for a late attack on a perceived weak spot in the Union line near Little Round Top, but were recalled by Lt. Gen. James Longstreet.

To his dying day, Bryan remained bitter, convinced that his men could have moved forward and won the battle.

[2] He was promoted to brigadier general on August 29, 1863, to replace the deceased Paul Jones Semmes and commanded the brigade during the Mine Run Campaign.

In September 1863, Bryan's Brigade traveled with James Longstreet to Georgia to reinforce the Army of Tennessee and they fought in the Battle of Chickamauga and the Knoxville Campaign.