The arboretum began as the private estate of businessman Jesse Tyson, who started construction of Cylburn Mansion in 1863.

The house, designed by Baltimore City Hall architect George Aloysius Frederick, was eventually completed in 1888 and remains intact, a stone structure built of gneiss from Tyson's quarries at Bare Hills, with mansard roof, tower, and an Italianate cupola.

Today the arboretum contains an extensive collection of trees and woody shrubs based loosely on the Tysons' original plantings.

The arboretum also includes a number of flower and vegetable gardens, as well as greenhouses designed and built in the 1960s by Lord & Burnham.

[3] It was used as a filming location for "Final Grades", a 2006 episode of The Wire, in which Bodie Broadus and Jimmy McNulty have a conversation in the park.