He started to build a palace with gardens on the hill of Puig del rey on the south of the town, the architects were Ramon Pau, Pons Descoll, and Bernat Quer with completion in 1309.
In 1415, Sigismund, Holy Roman Emperor, organised a European summit in Perpignan, to convince the Avignon Antipope Benedict XIII to resign his office and take to an end the Western Schism through the Council of Constance.
The Antipope refused to resign and to recognise the Pope that the Council had chosen, clashing with the Emperor, who left Perpignan on 5 November.
The General Council of the Pyrénées-Orientales bought the palace and the gardens in 1958, from the Ministry of Defence who keeps the Vauban bastions and 19th century buildings and courts as a garrison.
International guests includes the Rosenberg Trio, Tekamali and Paco Ibáñez in 2004, Montserrat Figueras, Rolf Lislevand and Manolo Sanlucar in 2005, The National and Sergio Lopez in 2006, Caetano Veloso in 2007, and the Rumberos Catalans, Bernardo Sandoval, Peter Finger and Aaron and Bryce Dessner in 2008.