The third of five children, he was baptized in his home four days later, on 10 November, by his paternal uncle, who was a canon by the same name, with the permission of the Naples curia.

Castaldo, influenced by Monsignor Francesco Morano and Father Luigi Maglione, decided to pursue a career in the Church, and attended the seminaries in Cerreto Sannita, Pozzuoli, and Naples.

Ordained a priest by Bishop Angelo Jannacchino on 8 June 1913, Castaldo served as a chaplain to the Italian Army (1915–1918), and a provost in Casoria (1918–1934).

Castaldo became Archbishop of Naples on 7 February 1958 and was created cardinal priest of S. Callisto[1] by Pope John XXIII in the consistory of 15 December that same year.

The Cardinal attended the Second Vatican Council from 1962 to 1965, and was one of the electors who participated in the 1963 papal conclave, which selected Pope Paul VI.