Isaac attended the University of North Carolina for one year in 1847, but left to manage a plantation for his father in Yancey County.

[1] Avery later formed a partnership with Charles F. Fisher and Samuel McDowell Tate to act as contractors in the building of the Western North Carolina Railroad in the mid-1850s.

With his state's secession from the Union, Isaac returned to Burke County, and with his brother Alphonso, recruited Company E of the 6th North Carolina Regiment.

With Hoke's wounding at the Battle of Chancellorsville in May 1863, Avery temporarily assumed command of the brigade in time for the Gettysburg Campaign.

Attacking in the early evening, Avery was struck in the neck by a musket ball and fell from his white horse, bleeding badly.

Early in his report wrote: "I had to regret the absence of the gallant Brigadier-General Hoke, who was severely wounded in the action of May 4, at Fredericksburg, and had not recovered, but his place was worthily filled by Colonel Avery, of the Sixth North Carolina Regiment, who fell, mortally wounded, while gallantly leading his brigade in the charge on Cemetery Hill, at Gettysburg, on the afternoon of July 2.

The Isaac E. Avery Chapter #282 of the Military Order of the Stars and Bars, a fraternal organization, is named in memory of the colonel.