Armistead was born in Upperville, Fauquier County, Virginia, and served as an orderly sergeant at the Battle of Fallen Timbers.

During the War of 1812, he was promoted to lieutenant colonel and successively served as Chief Engineer of the Niagara frontier army and the forces defending Chesapeake Bay.

His brother George Armistead commanded Fort McHenry during the Battle of Baltimore in the War of 1812.

The attack became immortalized by onlooker Francis Scott Key who penned "The Star-Spangled Banner" while watching the British bombardment of Armistead's fort.

His son Lewis Addison Armistead was a Confederate general who died during Pickett's Charge at the Battle of Gettysburg in 1863.