Graduating 37th of 40 from the United States Military Academy in 1822, Abercrombie's class included future Union Army generals Joseph K. Mansfield, David Hunter, and George A. McCall, as well as future Confederate Army general Isaac R. Trimble.

Seeing additional combat action as a captain during the Seminole Wars, Abercrombie was brevetted a major for gallantry at the Battle of Lake Okeechobee.

Serving on frontier duty throughout the 1830s and 40s within the South and Northwest, Abercrombie fought again during the Mexican–American War, where he gained distinction at the Battle of Monterrey and was promoted to lieutenant colonel.

They would have nine children, including Frank Abercrombie, who briefly played professional baseball for the Troy Haymakers of the National Association.

Promoted to brigadier general of Volunteers in the Union Army on August 31, 1861, Abercrombie commanded troops under his father-in-law Robert Patterson in the Shenandoah Valley and at the Battle of Hoke's Run.

Abercrombie commanded the troops that defended a depot in Fredericksburg, Virginia, in June 1864 against an attack by Hampton's Legion.