The 104th New York Infantry was organized at Geneseo, New York beginning in October 1861 and mustered in for three years service on March 4, 1862 under the command of Colonel John Rohrbach.

The regiment was attached to Wadsworth's Command, Military District of Washington, to May 1862.

1st Brigade, 2nd Division, III Corps, Army of Virginia, to September 1862.

1st Brigade, 2nd Division, I Corps, Army of the Potomac, to March 1864.

The 104th New York Infantry mustered out of service on July 17, 1865.