In 1574 the property that was then referred to as "Chilton Park" was split from older estates Calcot Manor and Chilton Foliat Manor by Edward Manners, 3rd Earl of Rutland who had inherited the property through his other title, the 14th Baron de Ros.

[1] The newly split Chilton Park was purchased by Sir Anthony Hinton (1532–1598) of Earlscourt (Earlscote) Manor outside of Wanborough, Wiltshire.

They sold it to John Zephaniah Holwell (1711–1798), a nabob, survivor of the Black Hole of Calcutta (1756), and the author of its most famous account.

Smith, the former commander-in-chief of the East India Company in Bengal, had retired in 1770, and reportedly returned to England with £300,000 (equivalent to £52,524,324 in 2023).

[7] Smith reportedly gambled much of his massive fortune away, and in 1785 was obliged to sell the estate to John Macnamara (1756–1818).

[8] In 1788, Macnamara sold Chilton back to the mortgagees William Morland (1739–1815) and Thomas Hammersley, bankers of Pall Mall.

[10] In 1796 the estate was purchased by John Pearse (1759–1836) whose family had been extremely successful in the fabric and clothing business, particularly in supplying the military.

[11] In addition to the family business, Pearse was on the board of the Bank of England, eventually becoming its governor in 1812, and was a Member of Parliament from 1818 to 1832.

[12] In 1800, Pearse had the roughly 7-year-old Soane-designed house pulled down and hired architect William Pilkington to design the current Chilton Lodge.

[18] For example, The Times notes that Major-General Randal Rumley died in an accident while visiting a Mr. William Butler at Chilton Lodge in 1884.

Since they were without children, Pearce's will directed that the Chilton estate be sold and the proceeds given to Trinity College, Cambridge.

The series' premise involved Dodson rehabilitating part of the property's largely derelict garden over a year.

The park was designed for John Pearse by Sir Humphry Repton at the same time the current house was built.

[23] [22] Interestingly, despite the demolition of the house he had designed, Sir John Soane appears to have been involved in Pearse's rework of the park and property.