Born in New York City, Baker's early education occurred there and in Boston.
[2] He studied business but turned to music as a career, becoming an organist in Concord, Massachusetts.
During his tenure he translated a considerable body of books and libretti into English, and wrote often in the Musical Quarterly, a Schirmer publication.
He published Baker's Dictionary of Musical Terms (1895) and most notably, Baker's Biographical Dictionary of Musicians (1900), which was revised after his death by Nicolas Slonimsky and then Laura Kuhn; as of 2007 is in its ninth edition.
He translated Oscar Paul's A Manual of Harmony For Use in Music-Schools and Seminaries and For Self-Instruction (1885) and numerous other works published by Schirmer.