He emigrated to Bermuda to teach and then relocated to New York City to take charge of a private school.

During the American Civil War, he enlisted in the 140th Pennsylvania Infantry in 1862 and rose through the ranks to be the regiment's colonel.

He fought at the Battle of Gettysburg in the Wheatfield area, taking charge of the 140th when all the senior officers were incapacitated.

He was taken as a prisoner of war during the Siege of Petersburg and incarcerated in Charleston, South Carolina.

In recognition of his service, on January 13, 1866, President Andrew Johnson nominated Fraser for appointment to the grade of brevet brigadier general of volunteers, to rank from March 13, 1865, and the United States Senate confirmed the appointment on March 12, 1866.