[6] The family also owned (Lower) Rooks Nest Farm until the early 1700s when it was acquired by the neighbouring Chesfield estate.

[8] One of the Howards sold the house in 1882 to Lieutenant-Colonel Robert Hindley Wilkinson, who lived at Chesfield Park in the neighbouring parish of Graveley,[6] Wilkinson already owned land in the hamlet of Rooks Nest as part of his Chesfield estate, and with the purchase of the house he then owned the whole hamlet.

[15] Forster was fascinated with the house and had affection for it,[16] seeing it as a secure home, though his mother never set down roots locally.

[17] Wilkinson died in 1888 and the property was inherited by his daughter, Caroline Poyntz-Stewart[18] Meanwhile, Lily became friends with the Poston family,[19][20] including Charles Poston, who moved into Highfield in Pin Green near Stevenage in 1886, with his wife, Clementine, son, Charles, and daughter, Mary.

[22] Forster was homesick so drew a sketch map of the house and garden, and wrote a brief "Rooks Nest memoir".

[24] Rooks Nest was then occupied by Dr Malcolm Williamson, the Australian-born Master of the Queen's Music,[27] who died in 2003.

[3] Forster's dislike of Wilkinson and the fact he made his mother unhappy led to him thinking all landlords unpleasant.

[30][31] Another memoir by Forster, from the 1940s and about West Hackhurst, Abinger Hammer in Surrey, which was his home from 1925, returns to the associations of Rooks Nest.

[32] Lily arranged for successive garden boys to play with Morgan on Wednesday afternoons but they were young and never stayed for long.

The main structure of the house is timber framed, and its front wall is of red brick with some grey headers.

The ground floor has a bressummer to a wide fireplace, the timber framework is exposed in some rooms and there are 18th century fielded panelled doors.

[11] However, the borehole did not benefit the Forsters and they did not have a well and they had to purchase drinking water from next door, the Franklin family at Rooks Nest Farm.

[39] Lily had the room photographed and a copy sent to a family member, in which she mentioned "two little black tea tables", "a tall, old-fashioned bureau [with] a small oak bookcase on each side of it" and a "piano opposite the window".

The landscape was termed 'Forster Country' in a letter to The Times signed by a number of literary figures, published on 29 December 1960, written in response to two compulsory purchase orders made by the Stevenage Development Corporation, and expressed the hope that 200 acres of countryside around the house could be preserved both as one of the last beauty spots within 30 miles of London and "because it is the Forster country of Howards End."

The letter says, "Literate people the world over feel that it [Forster country] should be preserved in its original setting as one of our greatest literary landmarks."

On 29 November 1997 a sculpture marking Forster's connection with the area was unveiled beside St Nicholas churchyard by the MP for Stevenage, Barbara Follett.