The regiment was composed of African American enlisted men commanded by white officers and was authorized by the Bureau of Colored Troops which was created by the United States War Department on May 22, 1863.

Colored Infantry was organized in Arlington, Virginia June 20 through November 11, 1863 and mustered in for three-year service.

The regiment of around 900 men arrived in Key West on February 22, 1864 as replacements for the 47th Pennsylvania Volunteers, where they encountered hostility from many of the locals.

The unit established a regimental headquarters and was encamped at Fort Taylor, under the jurisdiction of the Department of the Gulf.

Expedition from Cedar Key to St. Andrew's Bay July 20–29.