The house was built from 1870 to 1872 by Richard Norman Shaw for painter Frederick Goodall and named after the nearby prehistoric earthwork known as Grim's Ditch.

The house was designed in 1870 by Richard Norman Shaw for the Victorian era painter Frederick Goodall, who had purchased 110 acres (0.45 km2) of land[2] at Harrow Weald in 1856, but he did not begin to build until a lease on the property expired.

To the north of the house, Shaw built a small lodge, a walled garden and various outhouses and a stable block, later converted into garages by W. S. Gilbert for his collection of motorcars.

[3] Over the dyke (now a duck pond) Shaw built two stone bridges, which incorporated flint from the ruined church at Stanmore.

[6] He made various additions and alterations to the property, including an elaborate fireplace of Cornish alabaster in Goodall's studio, which became Gilbert's drawing/music room.

[9] At the house, Gilbert had a vinery, apiary, orchards and a farm, with Jersey cattle, horses, pigs and fowl.

[10][14] After Lady Gilbert's death, the contents of the house, apart from a few items kept by Nancy McIntosh, were sold at public auction on 17 and 18 March 1937, realising £4,600.

The house was acquired jointly by the Middlesex County Council and the London County Council, who leased it to the North West Regional Hospital Board from 1937 to 1962, at first as a rehabilitation centre for women suffering from tuberculosis (the house was used by the services during World War II).

By the early 19th century, the statue was described as being 'in a most wretched mutilated state; and the inscriptions on the base of the pedestal quite illegible'.

[18] Goodall placed the statue on an island in his lake at Grim's Dyke, where it remained while Gilbert owned the property.

Eventually it covered about one and a half acres, with an island in the middle, a punt house and changing hut, and an artificial waterfall that was ceremonially turned on in December 1899.

My pupil managed to struggle to the bank, and presently the gardener came and got out a boat, but it seemed a long time before they recovered the body.

[28] The family doctor, W.W. Shackleton, and Daniel Wilson of Bushey Heath Cottage Hospital, later certified that Gilbert had died at about 4.20 pm that afternoon of syncope (heart failure) brought on by excessive exertion.

The coroner's jury, also meeting in the billiard room at Grim's Dyke two days later, on 31 May 1911, recorded a verdict of accidental death.

[30] The dramatic architecture of this Victorian country house, and its typically English interior, have made it attractive as a film and television location.

[31] The following television shows and films, among others, used Grim's Dyke as a location: Listed building entries at Images of England for: