A child prodigy, he took over the duties of organist at the American Presbyterian Church in Montreal while still in his youth, serving in that capacity for eight years until 1858.

He ended his career as organist at the First Presbyterian Church in East Orange, New Jersey, where he served from 1895 until his death in 1915.

[1] In addition to his work as an organist and music editor, Warren was also a respected teacher of the organ.

One of his pupils, Augusta Maria Lowell (1857–1948), was one of the first women in North America to achieve success as an organist.

[1] In 2001 Dover Publications republished The Practical Organist (French: Organiste pratique), a work originally published by G. Schirmer in 1889 that encompassed 50 liturgical works for organ by the composer Alexandre Guilmant, for which Warren was the music editor.