Through his father's political connections, Rufus Ingalls was appointed to the United States Military Academy, graduating in the Class of 1843, which included his friend Ulysses S. Grant.

Ingalls distinguished himself in action at the skirmish at Eabudo and conflict at Pueblo de Taos, for which he received the brevet rank of first lieutenant.

In early 1854, while on duty in Washington DC, Ingalls was ordered to accompany the Steptoe Expedition from Fort Leavenworth in Kansas across the continent through the Utah Territory to the Pacific Coast.

Shortly after the First Battle of Manassas in July, he moved northward to Virginia to serve as aide-de-camp to Maj. Gen. George B. McClellan.

In June 1864, his old friend Ulysses S. Grant placed Ingalls in charge of supply with responsibility for all Federal armies operating against Petersburg and Richmond.

With the surrender of Robert E. Lee in April 1865 and the subsequent mustering out of much of the Union Army, Ingalls left the City Point depot in May.