He was a central figure in musical life in New Haven, Connecticut in the late 19th century, and is best remembered as the undergraduate teacher of Charles Ives while the composer attended Yale University.

[3] He finished his formal education in Europe, a common destination for a young American composer in the 1880s, where he studied in Munich with Josef Rheinberger.

[7] After his return to the United States in 1885, he was for two years professor of music in the Cathedral School of St. Paul in Garden City, Long Island.

[13] Bailey served for a time as a lawyer at the Securities and Exchange Commission having graduated from Yale in 1923 with majors in Greek and mathematics, but remained active as a composer for much of his life.

[14][15] Before leaving New York City in 1893, Parker had completed his oratorio, Hora Novissima, set to the opening words of De contemptu mundi by Bernard of Cluny.

It was widely performed in America; and also in England, in 1899 at Chester, and at the Three Choirs Festival at Worcester, the latter an honour never before paid an American composer.

His father, Charles Edward Parker, had been the architect for that congregation's chapel; famed Episcopal bishop Phillips Brooks laid the cornerstone.

[10] Parker's music is used as Hymn #66 "Rejoice the Lord is King"[16] in the hymnbook for The Church of Jesus Christ of Latter-Day Saints (with lyrics by Charles Wesley, 1707-1788).