He attended the public schools, and Hudson River Institute at Claverack, New York.

[1] During the American Civil War, he entered the Union Army as a first lieutenant and adjutant of the One Hundred and Seventy-seventh Regiment, New York Volunteer Infantry, and served in the Department of the Gulf in 1862.

After his service in the war, he graduated from the Albany Law School in 1864 and was admitted to the bar the same year.

After leaving Congress, Bailey was the United States consul to Hamburg, Germany, by appointment of President James Garfield, and served in that capacity from 1881 to 1885.

He served as a delegate to the Republican National Convention in 1888 and was appointed by President Benjamin Harrison as surveyor of customs at Albany, New York from 1889 to 1894.