He was ordained a priest on 27 March 1937 in the Patriarchal Lateran Basilica by Luigi Traglia, Vicegerent of Rome.

In 1956 Rossi was appointed Bishop of Barra do Piraí within the metropolitan district of Rio de Janeiro.

Paulo de Tarso Campos, bishop of his home diocese Campinas, consecrated him in the same year.

In the consistory of 1965 Rossi was created Cardinal-Priest by Pope Paul VI, and he received the title of Gran Madre di Dio.

In 1970 he was appointed Prefect of the Sacred Congregation for the Evangelization of Peoples, the recently renamed Sacra Congregatio de Propaganda Fide.