In the early nineteenth century the Ladbroke family were the principal landowners in Kensington, then a largely rural area on the western edges of London.

The Ladbroke Estate was located north of the Uxbridge Road (now Notting Hill Gate and Holland Park Avenue), and development of the land began in 1821, continuing until the 1870s.

He died childless, and around 1821 his land passed to his nephew James Weller, who assumed the name Ladbroke in order to be able to inherit.

The 1823 plan marks the genesis of Allason's most enduring idea—the creation of large private communal gardens enclosed by terraces and/or crescents of houses.

They had three children: Henry Weller Ladbroke Clarke (born about 1833, married 04 Jan 1855 Laura Frances Mary Agnes Morris "Florence Lancia", a famous operatic soprano of the times, and died after 1863 without children), Emily Georgina Clarke (born 25 Sep 1835, married 10 Nov 1857 Constanzo, Marquess del Carretto de Mombaldone, and had one daughter, Gertrude, who married Prince Emmanuele Gonzaga and left descendants) and Augustus Stanley Clarke (born 27 Feb 1837, died 1885 unmarried).