It was the creation of George, 1st Lord Lyttelton (1709–1773), secretary to Frederick, Prince of Wales, poet and man of letters and briefly Chancellor of the Exchequer.

Before the death of his father (Sir Thomas Lyttelton) in 1751, he began to landscape the grounds in the new Picturesque style, and between 1754 and 1760 it was he who was responsible for the building of the Neo-Palladian house that survives to this day.

After a fire in 1925, most of the house was restored, but the uppermost floor of the servants' quarters was not, which means that the present roof line between the towers is lower than it was when first constructed.

His brother and successor Christopher Charles Lyttelton, 12th Viscount Cobham began restoration works in both the main house and the park.

The two finest examples of this style in Worcestershire were Croome Court built between 1751 and 1752 and Hagley Hall designed by Sanderson Miller (with the assistance of the London architect John Sanderson) between 1754 and 1760. Notable Neo-Palladian features incorporated into Hagley Hall include the plain exterior and the corner towers with pyramidal roofs (a feature first used by Inigo Jones in the design of Wilton House in Wiltshire), and of Venetian windows.

[3] The house contains a fine example of Rococo plasterwork by Francesco Vassali and a unique collection of 18th-century Chippendale furniture and family portraits, including works by Van Dyck, Joshua Reynolds, Cornelius Johnson, and Peter Lely.

A short walk to the west of the Hall, facing its rear facade, is the parish church of St John the Baptist, and its surrounding churchyard.

[10][11] The grounds drew many admiring visitors, including other writers interested in landscaping such as Alexander Pope and William Shenstone, to both of whom monuments were later erected in the park.

James Thomson was another commemorated visitor, who included a description of the grounds in the Spring section of The Seasons, which he revised following his first visit to Hagley in 1743.

[13] Horace Walpole, notoriously hard to please, wrote after a visit in 1753, "I wore out my eyes with gazing, my feet with climbing, and my tongue and vocabulary with commending".