On the latter's death it passed directly to his grandson Henry Reveley Mitford (1804–1883), whose father had been drowned at sea in 1803 after his ship hit Bell Rock.

It was sold to Major John Forster in the early 1880s, but after his death in 1886, it was let to Bertram Freeman-Mitford, 1st Baron Redesdale, and his wife, Lady Clementine (née Ogilvy) in the 1880s.

He died in 1942 and the house was requisitioned by the Royal Navy as a headquarters to prepare for Operation Neptune.

In the novel, Shute identifies Mastodon as Exbury, and describes the wonder of Prentice and a fellow Wren when they first arrive at the grand river-front house and explore its gardens.

Among other things, they find underground irrigation systems, carefully labelled plants, and "... a rock garden half as large as Trafalgar Square that was a mass of bloom ..." All of this, says Shute, was tended by a gardening staff that "... had been reduced from fifty to a mere eighteen old men."