Born at the Palace of Portici outside Naples, he was named Gabriel Antonio Francisco Javier Juan Nepomuceno José Serafín Pascual Salvador; he was the fourth son of King Charles VII and V and Maria Amalia of Saxony; his father was the King of Naples and Sicily as part of a personal union from 1735.

He had Antonio Soler as his music teacher, who composed several sonatas on harpsichord especially for his gifted pupil, as well as concerts for two organs to be interpreted together in the El Escorial church.

Gabriel spent his childhood growing up in his father's Neapolitan kingdom; at the age of seven, he moved with his parents and older siblings Infante Charles and Infanta Maria Luisa to live in Madrid.

In Naples he left his two other brothers Infante Philip, Duke of Calabria (who was disabled and thus excluded from succession to any of his father's domains) and the eight-year-old Ferdinand, King of Naples and Sicily; Ferdinand was put in the care of a Regency council that ran the Kingdom till 1767.

While in Spain his brother married Maria Luisa of Parma in 1765; the couple did not have any issue until 1771; Gabriel was third in line to the Spanish Throne during that period.