In 1892, Gunn immigrated to the United States to teach moral theology at the Marist House of Studies at The Catholic University of America in Washington, D.C.[1] In 1898, Gunn was assigned to Atlanta, Georgia, to become pastor of the newly erected Sacred Heart Parish.

[1] During his pastorate at Sacred Heart, he also founded and served as the first president of the Marist College, and established a parochial school.

[3] He received his episcopal consecration on August 29, 1911 from Archbishop James Blenk, with Bishops Edward Allen and John Morris serving as co-consecrators, at Sacred Heart Church.

[4] He then began extensive pastoral visits to all the parishes and missions throughout the diocese, which covered nearly 47,000 square miles.

[1] In his will, Gunn states, "In life and in death I am proud of three things: My Irish birth, my Catholic faith, and my American citizenship.