As a university professor he had been active in the state militias of Kentucky and Tennessee and on the outbreak of hostilities he sided with the South, despite having been born in the North into a family of abolitionist Quakers.

After the start of the Civil War, Johnson entered the service June 28, 1861, as a colonel of engineers in the Tennessee Militia, and a week later this commission was changed to be in the Confederate States Army.

He commanded a division of the army at Donelson, but was effectively overshadowed by the more politically astute Pillow, who led the wing in a fierce assault in an attempt to break out and escape from the encircled fort.

Gen. Ulysses S. Grant on February 16, 1862, but two days later Johnson was able to walk unimpeded out through the porous Union Army lines and escaped capture.

At the Battle of Chickamauga, Johnson's brigade spent September 18 on the Confederate right, assigned to the command of John Bell Hood's division of James Longstreet's corps, then just arriving from Virginia.

After spending the winter of 1863–64 in northeastern Tennessee, Longstreet's force was transported by rail back to Virginia to reinforce Robert E. Lee for the Overland Campaign.

En route, Johnson alone was diverted to Petersburg, to command a division in the Department of North Carolina and Southern Virginia under P. G. T. Beauregard.

The mine was set off under part of Elliott's South Carolina Brigade, which rallied and captured three stands of colors and 130 prisoners that day.

When Beauregard was transferred to the western theater in October, Johnson's division was assigned to Anderson's Fourth Corps in the Army of Northern Virginia under Robert E. Lee.

He was originally buried in Miles Station, near Brighton, but was reinterred in 1975 to Old City Cemetery, Nashville, Tennessee, to be next to the grave of his wife, Mary.