Gutstein was born to Naftali and Sarah Pearl Taubes in Otynia, a small town in the province of Galicia (Central Europe), which at the time of his birth was part of the Austro-Hungarian Empire.
"[2] He was instrumental in the effort to obtain recognition for the Touro Synagogue as a national historic site, a designation that was bestowed by Congress in 1946.
He continued his graduate studies concurrently, obtaining Orthodox ordination from Rabbi Eliezer Lipa Weisblum in 1937 and a Ph.D. in history in 1939 from Webster University.
Under his leadership, Congregation Shaare Tikvah grew to become one of the largest Conservative Jewish synagogues in the Chicago metropolitan area.
He authored numerous books and articles including A Priceless Heritage, a history of the first 100 years of Chicago Jewry, collections of essays entitled To Bigotry No Sanction and Profiles of Freedom, and sermonic discourses in Frontiers of Faith.
Dr. Louis Finkelstein, Chancellor of the Seminary, said to him in awarding him his honorary Doctor of Divinity, "You have added one more jewel to the crown that is your family's name."