Ayres was born at East Creek, New York, along the Mohawk River in Montgomery County.

He graduated from the United States Military Academy in 1847, ranking 22nd in a class of 38, and was commissioned a brevet second lieutenant in the 4th U.S.

Between the wars, Ayres was stationed at various posts on the frontier and served at the Fort Monroe Artillery School from 1859 to 1861.

At First Bull Run, his battery, attached to the brigade of William Tecumseh Sherman, was held in reserve and he did not see action during the battle proper, but distinguished himself by providing cover for retreating Union Army troops pursued by Confederate cavalry.

Just before the Battle of Fredericksburg, he was promoted to chief of artillery of the VI Corps as a brigadier general, as of November 29, 1862.

While recuperating from an injury caused when his horse fell, Ayres considered his military career and realized that artillery officers had a much slower rate of promotion than their colleagues in the infantry.

Due to a great Confederate assault nearby at the Peach Orchard, Caldwell's division retreated, because his two brigades were at risk of being surrounded.

As part of the general reduction of ranks that was typical following many American wars, Ayres returned to the regular army with the rank of lieutenant colonel and he performed mostly garrison duty in a number of posts in the South, including Little Rock, Arkansas, Jackson Barracks, New Orleans, Louisiana, and Key West, Florida.

Ayres died on duty on December 4, 1888, in Fort Hamilton, New York, at 62 years of age.