The building that became Walpole House was built late in the Tudor era; internal features survive from the 16th and 17th centuries.
[6] In front of the house is an elegant[5] Grade II* listed screen and wrought iron gate;[5] the brick gateposts are topped with white globes, ball finials.
The lower lawn is bordered by raised curved flowerbeds edged with flagstones; to the west of the stone path is a large mulberry tree.
[5][7][8][9] The back boundary wall is Grade II listed; it is described as "of plum-red brick about 9-10 ft high" with "some black headers".
The listing states that it is the remaining part of the boundary of College House, where the scholars of Westminster School came to escape the plague starting in 1557.
[10] Walpole House was the last home of Barbara, 1st Duchess of Cleveland, a notorious royal mistress of King Charles II, until her death in 1709; she is buried in St Nicholas Church nearby.
[5] Walpole House most likely provided the model for the fictional academy for young ladies in his 1847–48 novel Vanity Fair,[6] which begins with the words "While the present century was in its teens, and on one sunshiny morning in June, there drove up to the great iron gate of Miss Pinkerton’s academy for young ladies, on Chiswick Mall, a large family coach, with two fat horses in blazing harness, driven by a fat coachman in a three-cornered hat and wig, at the rate of four miles an hour.