Born in Ingolstadt, Joseph Schröffer studied at the seminary in Eichstätt and the Pontifical Gregorian University in Rome before being ordained to the priesthood on October 28, 1928.

Schröffer attended the Second Vatican Council from 1962 to 1965, and entered the Roman Curia upon being named Secretary of the Sacred Congregation of Seminaries and Universities on May 17, 1967.

As Secretary, he was the second-highest official of that dicastery, successively under Cardinals Giuseppe Pizzardo and Gabriel-Marie Garrone.

In 1973, following the release of the Congregation for the Doctrine of the Faith's document Mysterium Ecclesia, the Archbishop attacked the liberal theologian Hans Küng for his opposition to papal infallibility.

The Cardinal served as a special papal representative to the jubilee celebration of the Cologne Cathedral on August 15, 1980.