When the Civil War erupted, Indiana began organizing similar militia groups and integrating them with companies of new recruits into full regiments.

He served in what became West Virginia at the battles of Cheat Mountain and Greenbrier River in late 1861, then participated in several fights against troops of Stonewall Jackson during the Valley Campaign as lieutenant colonel of the 14th.

Serving in the Army of the Potomac at Antietam, Harrow's regiment attacked enemy positions along the Sunken Road, where half his men became casualties.

[4] In April 1863, Harrow was finally promoted to brigadier general and led the 1st Brigade, 2nd Division, II Corps, during the Gettysburg Campaign.

Most notably the 1st Minnesota Volunteer Infantry distinguished itself in stalling a Confederate attack on an under-defended part of the Federal line.

Defending a portion of the Union line on Cemetery Ridge during the climatic fighting on July 3, Harrow's men helped repel a part of Pickett's Charge.

Harrow was reassigned to the Western Theater, where he participated in the Atlanta Campaign in command of the 4th Division of the XV Corps, a change regretted by those soldiers.

His men initially resented him, as "Western officers and soldiers held a sort of prejudice against the Eastern army," in which Harrow had most recently served.

Compounding the problem, "General Harrow was far from possessing the necessary qualifications to induce men to obey him ..." and instituted a series of strict and unusually harsh disciplinary measures.

Perhaps due to his ill temper and past reputation, a number of high-ranking generals, including Oliver O. Howard, William T. Sherman, and Winfield S. Hancock, refused to accept him.