Consequently, he spent his childhood under the guardianship of William de Manoir in the cities of Aix, Sisteron, St Victor, near Marseilles, and Barjols.

It is from Sisteron that a letter was sent, dated 2 May 1286, by him and his two elder brothers, Robert and Saint Louis of Toulouse, asking King Edward I of England for the release of their captive father.

The three princes were treated with the honour due their status and they carried an entourage of a hundred young noblemen.

Raymond invited Pierre de Jean-Olieu to visit him in prison, but received only a letter of consolation.

In December 1300, Raymond received the honours of Monte Sant'Angelo, Capaccio, Eboli, Isernia, Atri and Vieste, and Altamura, the counties of Gravina and Andria, and the castellanies of Vairano, Lesina, and Terra di Muro.