[1] The first house on the Claremont estate was built in 1708 by Sir John Vanbrugh, the Restoration playwright and architect of Blenheim Palace and Castle Howard, for his own use.

At the same time, he built the stables and the walled gardens, also probably White Cottage, which is now the Sixth Form Centre of Claremont Fan Court School.

From this so-called "prospect-house", or belvedere, he and his guests could admire the views of the Surrey countryside as they took refreshments and played hazard, a popular dice game.

Lord Clive decided to demolish the house and commissioned Capability Brown to build the present Palladian mansion on higher and dryer ground.

The estate then passed through a rapid succession of owners; first being sold "for not more than one third of what the house and alterations had cost"[5] to Robert Monckton-Arundell, 4th Viscount Galway, and then to George Carpenter, 2nd Earl of Tyrconnell, and finally to Charles Ellis, 1st Baron Seaford.

In 1816, Claremont was bought by the British Nation through an Act of Parliament as a wedding present for George IV's daughter Princess Charlotte and her husband Prince Leopold of Saxe-Coburg.

"[7] To the nation's great sorrow, however, Princess Charlotte, who was second in line to the throne, was, after two miscarriages, to die there after giving birth to a stillborn son in November the following year.

She, in turn, lent the house to the exiled French King and Queen, Louis-Philippe I and Maria Amalia (the parents-in-law of Leopold I of Belgium), after the Revolutions of 1848.

[10] In 1870, Queen Victoria commissioned Francis John Williamson to sculpt a marble memorial to Charlotte and Leopold which was erected inside the house.

)[12] Victoria bought Claremont for her fourth, and youngest, son Prince Leopold, Duke of Albany, when he married Princess Helena of Waldeck and Pyrmont in 1882.

Claremont should have passed to Charles upon his mother's death in 1922, but because he served as a German general in the First World War, the British government disallowed the inheritance.