[2] In 1941, Carlebach's family was deported along with the entire Jewish community of Hamburg to the Jungfernhof concentration camp near Riga in Latvia.
Carlebach was serving as a high school teacher of Torah studies and Talmud in the Yeshiva of Eastern Parkway in Brooklyn, when Hutner appointed him the new mashgiach ruchani of Chaim Berlin following the departure of Avigdor Miller in 1964, who had been long tenured in that position.
He served as mashgiach of the yeshiva and Kollel Gur Aryeh (its post-graduate division) from 1966 to 1978, after which he was succeeded by Shimon Groner, one of Hutner's trusted disciples.
He made aliyah together with his only daughter, Bruria David, who was childless, leaving Yeshiva Rabbi Chaim Berlin and Kollel Gur Aryeh to his disciples.
The result of this power struggle was Carlebach being denied access to the yeshiva, though he subsequently refused to relinquish the title mashgiach ruchani.
Carlebach attempted to bring the termination of his employment and the manner in which it was done to adjudication with various batei din (Jewish religious courts).
Following his departure from Chaim Berlin, Carlebach delivered lectures at various Beis Yaakov schools and seminaries for young women.
His eldest daughter, Elisheva Carlebach is the Salo Wittmayer Baron Professor of Jewish history, culture and society at Columbia University.