The regiment was composed of African American enlisted men commanded by white officers and was authorized by the Bureau of Colored Troops, which was created by the United States War Department on May 22, 1863.

Colored Infantry was organized at Camp William Penn near Philadelphia, Pennsylvania beginning January 10, 1864 for three-year service and mustered under the command of Colonel Joseph Barr Kiddoo.

1st Brigade, Hincks' Division (Colored), XVIII Corps, Army of the James, to June 1864.

The expedition continued as follows: an expedition to King and Queen County March 9–12; Butler's operations south of the James River and against Petersburg and Richmond May 4-June 15; duty at Wilson's Wharf, James River protecting supply transports, then constructing works near Fort Powhatan until June; the attack on Fort Powhatan May 21; before Petersburg June 15–18; siege operations against Petersburg and Richmond June 16, 1864, to April 2, 1865; Deep Bottom August 24; Dutch Gap August 24; demonstration north of the James River September 28–30; Battle of Chaffin's Farm, New Market Heights, September 29–30; Fort Harrison September 29; Battle of Fair Oaks October 27–28; Chaffin's Farm November 4; in the trenches before Richmond until April 1865; occupation of Richmond April 3; moved to Washington, D.C., and participated in the obsequies of President Lincoln, and afterward to the eastern shore of Maryland and along lower Potomac in pursuit of the assassins.

They then rejoined the corps May 1865 and moved to Texas May 24-June 6 for duty along the Rio Grande until October 1865.