[1] Elm Farm is located in rural northern Danville, on the west side of Main Street just south of its junction with Beach Plain Road.

The property consists of 85 acres (34 ha) of land, stretching in a relatively narrow strip all the way west to the town line with Sandown.

The westernmost portions of the property consists of woods and swamp, while the 30 acres (12 ha) immediately west of the road are given over to crops, pasture, and the farmstead.

[2] The property has been in agricultural use since at least 1835, when Samuel Sargent Jr. purchased part of the 7th lot of the town's original land grants made in 1760.

His son Bailey operated a variety of businesses, including a cider mill, and offered stabling services.