It was originally the London townhouse of the Dukes of Argyll, a prominent Scottish family, before later passing into the hands of the nineteenth century politician Lord Aberdeen after considerable redevelopment.
John Campbell, 2nd Duke of Argyll, known for his role in the Act of Union and the 1715 Jacobite Rising, acquired a property in 1706 and used it as his London residence along with Sudbrook Park to the west of the capital in Surrey.
The designer of the house is unknown but James Gibbs, who helped on the development of the nearby Argyll Street for the family, may have assisted with the exterior of the building.
Aberdeen commissioned the architect William Wilkins to rebuild the house at considerable expense in the fashionable Greek Revival style.
[5] In 1814, during the Allied sovereigns' visit to England, the Austrian foreign minister Klemens von Metternich based himself there at the invitation of Aberdeen who had recently served as British Ambassador to Austria.