In the 15th century, Henry IV of Castile built the first hunting lodge on the site, along with a small shrine dedicated to Saint Ildephonsus of Toledo, which gave this place its first name.

Isabella I of Castile granted both buildings to the monks of the Santa Maria del Parral monastery in Segovia, who built an almshouse and developed a granja (farm) alongside the lodge and shrine.

For the architect, Philip began with an unpretentious project by the Spanish architect Teodoro Ardemans, incorporating a chapel centred on one facade, which was enlarged in a second phase, ca 1728-34 under Andrea Procaccini and Sempronio Subisati, who provided the courtyards in the flanks, then given its definitive character by Filippo Juvarra, who was brought from Turin, based on recommendations in the circle of Philip's second queen, Elizabeth Farnese of Parma,[1] and his assistant, Giovanni Battista Sacchetti.

The glass factory, which had some initial successes from 1720 at Nuevo Baztan in the province of Madrid, was moved under the direction of its Catalan foreman, Ventura Sit, to San Ildefonso, where supplies of timber were plentiful, and a royal patron was near.

[4] Philip's successor Ferdinand VI bequeathed the royal site of San Ildefonso, with all it contained, to his father's second wife, Elisabeth Farnese, who was effectively forced to live there, well away from Madrid politics, for the duration of his reign.

For the next one hundred and twenty years, La Granja was the court's main summer palace, and many royal weddings and burials, state treaties, and political events took place within its walls.

[5] It is a popular tourist attraction, with gardens, and interiors displaying rooms with Carrara marble, Japanese lacquerware, and crystal chandeliers; portraits and other paintings; and a Museum of Flemish tapestries.

A group of richly sculptural vases have been attributed to designs by the "dazzling maverick" Gilles-Marie Oppenord,[9] which were probably forwarded through the offices of Robert de Cotte, overseeing French royal building projects as intendant des Bâtiments du Roi.

Bruno Pons noted in the sculptural vases "an almost excessively brilliant style, quite distinct from French royal taste and showing an undeniably superior understanding of ornament".