The earliest local records of the Ashlyn name occur in a document dating from 1314 which describes a Reginal Asselyn, who gave his name to a house lying east of Brickhill Green.

The old house that stood on the site of the present hall was sold in 1764 along with it garden, 200 acres of farmland, two orchards and additional forest land.

The houses were all in close proximity to a major coaching route along Akeman Street, offering convenient access to London.

[5] Robert Algernon Smith, the younger brother of Augustus, married into another banking family, the Dorriens who lived at neighbouring Haresfoot house.

[10] Another section of the estate, the former kitchen garden to the West of the hall, was sold off to establish Ashlyns Care Home in around 2000.

On the south-western side, the house features a semi-circular bowed front with a first-floor verandah of wrought iron overlooking ornamental gardens.

[14] The surrounding estate consists of pasture fields and parkland that features large, mature wellingtonia and cedar trees.