He is considered an important figure in the development of the 20th-century early music revival and adoption of historically informed performance practices in the United States.

He studied conducting with Robert Shaw, G. Wallace Woodworth, William Ifor Jones, and Anthon van der Horst.

[3] Harold Schonberg assessed a 1959 concert of works by Purcell and Britten with these words:[1] In Mr. Dunn the Cantata Singers have an alert, exciting conductor whose rhythm is impeccable and who has immediate control over his forces.

With that group in 1964 he led the New York premiere of Haydn's C Major Cello Concerto, a work discovered in 1961, with Janos Starker.

During his tenure there, he transformed the organization from a traditional amateur oratorio society to a fully professional ensemble with a repertoire that ranged from early music to modern works.

He composed works suitable for small church choirs, described by their publisher, Cantate Music Press, as "adaptable to the exigencies and emergencies that are apt to befall Sunday mornings."