He first studied music with his father; later his teacher was another Hungarian composer, Mihály Mosonyi.
He started his career just as an orchestra musician of the National Theatre in Pest playing timpani; he was also the dulcimer soloist at the premiere of his father's opera, the Bánk bán.
Later he became the conductor of the theatre's orchestra, and at the age of thirty in 1876, he was appointed the music director of the ensemble.
Erkel accepted the chairman-conductor position of the Budapest Philharmonic Orchestra in 1875, a post he held until 1900.
As a composer his oeuvre is relatively small, including orchestral and choral works.