[5] The founding of Clarke School for the Deaf in 1867[6] and the founding of Smith College in 1871 marked the beginning of a significant number of institutional buildings in a wide range of architectural styles at the eastern end of the district (Smith College) and Round Hill Road (Clarke Schools).

In 2013 the Clarke Schools sold 12 acres with ten buildings to a developer who will create luxury condominiums and commercial space.

The eastern terminus of the Elm Street Historic District is dominated by Smith College's College Hall on the south side built in 1875 in Gothic Italianate style by Peabody and Stearns[8] and by the former St. Mary's Catholic Church built on the north side in 1881 in Gothic Revival.

The Church and its Rectory are on the site of the former Mansion House, an important stop at the northern terminus of the Northampton-New Haven Canal.

It is believed to have been built around 1725 by Jonathan Hunt and would therefore be the oldest building owned by Smith College.

[15] The western terminus of the District is Catholic Church built in 1899 for the Blessed Sacrament Parish in an unusual "Swiss" design.

Elbridge Southwick owned and developed Bedford Terrace (now largely Smith College) in the 1890s and the office block known as Central Chambers on Center Street, Northampton.

Classical Revival example includes: 68 Elm St., known as John M Greene Hall, Smith College.

Romanesque Revival example includes 48 Elm St., St John's Episcopal Church shown 1905.

Swiss style example includes 354 Elm Street, the Catholic Church at western end of Historic District.