He is also remembered for his close friendship with and influence upon future Supreme Court justice Oliver Wendell Holmes Jr., who was his classmate at Harvard and his comrade during the war.

[6] Hallowell fought in the Battle of Ball's Bluff on October 21, 1861, in which he distinguished himself by leading a line of skirmishers to hold off Confederate forces.

[10] On April 17, 1863, he was promoted to lieutenant colonel, as second-in-command (after Colonel Robert Gould Shaw) of the 54th Massachusetts, one of the first all-black regiments in the U.S.[8] In a letter to the governor of Massachusetts, John Murray Forbes recommended Hallowell for the promotion based on his bravery, soundness of mind, and willingness to lead a black regiment despite the fact that many found the idea "unpopular.

"[11] On May 30, he accepted Governor John A. Andrew's personal request that he be made colonel in command of the 55th Massachusetts, another all-black regiment.

Hallowell had been his "oldest friend...[and was] the most generously gallant spirit and I don't know but the greatest soul I ever knew....[H]e gave the first adult impulse to my youth.

In the Civil War film, Glory, Norwood and his brother were recreated as the fictional character, Major Cabot Forbes, portrayed by actor Cary Elwes.