[1] The future district was first established by its oldest contributing property, the Ten Broeck Mansion, built at the end of the 18th century by a prominent local family.

In the late 2010s Albany mayor Kathy Sheehan and her husband bought and rehabilitated one of the district's vacant townhouses to make their home.

Its southern corner is Sweet Pilgrim Baptist Church at the junction of Ten Broeck Street and Clinton Avenue (part of U.S. Route 9), a block west of the Palace Theatre and across from Albany's Family Court building.

There, the Ten Broeck Mansion, originally known as Arbor Hill, which later became the name for the district,[4]: 14  occupies a large parcel at the intersection's southwest corner.

These include Bethany Baptist Church, St. Joseph's Youth Center and some commercial properties along the adjacent blocks and the public basketball courts at the Second Street intersection.

On the west of North Swan are similar neighborhoods but with newer, less coherent architecture (One building near the district, the Stephen and Harriet Myers House, is listed on the Register).

[7]: 5  To the east, downhill to North Pearl Street (New York State Route 32), are neighborhoods of mixed older and newer buildings and vacant lots, slowly being redeveloped, closer to the Hudson River.

The next year, Stephen van Rensselaer II, the patroon of his family's lands to the north of the city in what was then the town of Watervliet,[13] including the future district, began having them surveyed.

By 1797, when he and his wife Elizabeth built the brick Federal style mansion on five acres (2 ha) that they leased from her brother, Stephen van Rensselaer III, he had served as a delegate to the Second Continental Congress in 1775, a general commanding New York's militia at the Battle of Saratoga and later Albany's mayor and a state senator.

[15] Ten Broeck named the mansion Prospect due to the commanding view it offered of the Hudson and its river traffic (some other accounts suggest he called it Arbor Hill from the beginning).

Some found the mansion neighborhood and the blocks to its west to their liking, enough that city directories of the period show a few residents and businessmen listing "Arbor Hill" as their address.

[4]: 14  In 1848, the neighborhood regained some of its original prestige when Thomas Worth Olcott, a prominent banker, bought the Ten Broeck Mansion and formally renamed it Arbor Hill.

[15] The King Building, an elaborate Greek Revival duplex, was built over four years in the 1850s at 27–29 North Swan in front of the archway that provided access to the mansion's grounds on the rear.

[4]: 13 In 1855 St. Joseph's Church was built on Ten Broeck between First and Second to serve the growing Irish immigrant population, which generally did not live in the district but near it, close to their jobs along the riverfront.

At the north corner, the polygonal home at 105 Ten Broeck featured an iron mansard roof, a hallmark of the contemporary Second Empire style.

They rented the houses they left behind or sold them, with the result that many were subdivided into multiple units, making the Ten Broeck Triangle less exclusive than it had been for the previous 50 years.

Together with the nearby house at 3 St. Joseph's, a one-story brick building with marble trim and a Spanish-style pantile roof, built that same year, they are the latest-built contributing properties to the district.

After a century of ownership, the descendants of James Olcott Worth formally transferred the Ten Broeck mansion from their family to the Albany County Historical Association.

[5]: 16 In 1975 several residents of the Ten Broeck Triangle formed the Arbor Hill Concerned Citizens Association (AHCCA) to lobby the city for improvements in neighborhood services.

[2] The city's urban renewal agency helped to repopulate the area with three auctions of foreclosed houses on Ten Broeck Street during that period.

[16]: 102–103 Continuing historic surveys and inventories of the area found that the blocks between the Ten Broeck Triangle and North Swan also had many intact rowhouses from the same 19th century period with little modern intrusion, even if some of them were in a state of neglect and disrepair.

[24] A plan to rehabilitate it for use as a nightclub fell through, and when the building itself was on the verge of collapse in the winter of 2001 the city had to shore it up and take possession under eminent domain.

[25][26] It was in turn deeded to Historic Albany, which was able to secure matching funds for the necessary work[27] which continued to stage cultural events there to raise all the money for a more complete restoration.

[30][31] The AHCCA founded a separate group called the Arbor Hill Improvement Corporation in the early 1980s to address housing issues that the private developers of the area were not.

[16]: 116 In 2002 local artist Yacob Williams painted the first of four murals in the Arbor Hill neighborhood on the house at the corner of North Swan and Livingston, a project intended to improve the quality of life in a high-crime area.

Restoring the house turned out to be more extensive than they expected, requiring replacing a side wall, the roof, the entire plumbing and electrical systems, and gutting the interior.

The Ten Broeck Triangle, it noted, was one of three areas of Arbor Hill where property values did not suffer from the neighborhood's bad reputation.

"Clearly, to the extent that neighborhood-wide problems are alleviated or solved, property owners in the Ten Broeck neighborhood will greatly benefit and market values can be expected to escalate rapidly.

[37]: 17–23 In a 2005 progress report on the plan, the city noted that the state had awarded the Albany Housing Authority a grant for a feasibility study on the former St. Joseph's School.

[38] Four years later the national APA recognized the entirety of Arbor Hill as one of its 10 great neighborhoods in America, along with Washington, D.C.'s Adams Morgan, Savannah, Georgia's Victorian District and Seattle's Fremont.