Baumgarten immigrated to the United States with his family in 1939 as a result of the Anschluss, Germany's occupation of Austria in 1938.
It was a chance meeting at Johns Hopkins with William Foxwell Albright that caused him to change direction, eventually being awarded a Ph.D. in Semitic studies in 1954.
He was a fellow at the Annenberg Institute (now the Katz Center for Advanced Judaic Studies of the University of Pennsylvania) in Philadelphia in 1992–1993.
His knowledge of Qumran legal matters was probably the reason why John Strugnell gave him the task of publishing the Cave 4 fragments of the Damascus Document.
His work on the Damascus Document not only brought its manuscripts to publication but provided the basis for understanding it within the framework of the history of Judaism.