He graduated second in the Class of 1862 at the United States Military Academy and was commissioned a 2nd lieutenant in the Corps of Engineers on June 17, 1862.

He commanded two companies of the engineer battalion which built fortifications and pontoon bridges throughout the Virginia campaigns until General Robert E. Lee's surrender at Appomattox in April 1865.

On October 27, 1897, Gillespie received the Medal of Honor for carrying dispatches through enemy lines under withering fire to Major General Philip Sheridan at the Battle of Cold Harbor, Virginia, on May 31, 1864.

After the Civil War, Gillespie successively supervised the improvement of harbors at Cleveland, Ohio, Chicago, Boston, and New York City.

He initiated construction of the canal at the Cascades of the Columbia River and built the famous Tillamook Rock Lighthouse off the Oregon coast.

He had charge of ceremonies at President William McKinley's funeral and at the laying of the cornerstone of the Army War College building in 1903.

General Gillespie's design entirely changed the planchet to feature the head of Minerva in the center of a star surrounded by a wreath.

Gillespie's final assignment was as Assistant Chief of Staff of the United States Army from 1904 to 1905 with the rank of major general.