In the mid-20th century it began to decline when the construction of Empire State Plaza cut it off from downtown and adversely impacted the neighborhood.

The South Mall Expressway, carrying traffic from the Dunn Memorial Bridge to Empire State Plaza separates the district from the Times Union Center and downtown to its north and northeast.

Erastus Corning Tower and the other modernist high-rises of Empire State Plaza loom over the district from the northwest.

Along its west is a strip of land that includes the governor's mansion, Cathedral of the Immaculate Conception and the New York State Library.

The north side is delineated on the corners by Madison Avenue (US 20) but there the boundary includes most of Van Zandt Street and most of the south side of Hamilton before returning to Madison along Philip Street and the rear lot lines of the properties at the Van Zandt corner.

[1] Charles, Elm, Myrtle and Wilbur streets are included in their entirety, as are Ash Grove, Bleecker and Madison places.

In its earliest days of settlement, it was known as Albany's first suburban enclave, an area where the affluent built large homes and surrounded them with gardens and landscaping.

Italians, the last of several immigrant groups to make Mansion their first home in Albany, became identified with it after a huge influx in the late 19th century.

At the time of Albany's settlement in the 17th century, the city was concentrated in a stockaded area roughly equivalent to its present downtown.

When he died in 1766, his family subdivided the area and sold the lots to wealthy Albanyites who built large houses for themselves.

Yates spent his later years in the house, selling off the portion north of Ashgrove and Westerlo streets around the same time the rowhouses were being built on Trinity.

Eleven years later, the academy moved to its current quarters and Kane's estate was demolished to make way for a church.

The pointed-arch windows and Gothic tracery suggest the possible influence of Andrew Jackson Downing's architectural theories.

[1] Another major builder of the Mansion District was James Eaton, superintendent of building at the New York State Capitol for some time during this period.

[5] He also built the Gothic Revival 78–92 Grand Street, with a roof finial dividing its symmetrical combined facades.

[1][4] During this era the district became home to a large population of German immigrants, many of them Jews fleeing the collapse of the Revolution of 1848.

They built the Italian Renaissance Revival St. Anthony's Church at Grand and Madison in 1908, and at the next census two years later they had become the fourth largest ethnic group in the city of Albany.

The organization persisted, and has been credited with helping the neighborhood rebound in the early 21st century (although some residents complain that neglect continues in some areas, and arson destroyed several houses on Madison Place in 2007[10]) The Albany Free School, a democratic school founded on Elm Street in 1969,[11] also became one of its key institutions.

Houses in the area sell well due to their walkable proximity to the Plaza and downtown, and newer arrivals praise the neighborhood's sense of community.