[4]The original 164 acres (0.66 km2) of the park were presented as a gift to New York State by Robert and Laura Treman in 1924.

[citation needed] The rock formations within the park are primarily made of Devonian shale and sandstone.

[7] These rocks formed in horizontal layers, creating flat slabs and angular edges as they erode and fracture.

The gorge and waterfalls formed since the last ice age — within the last twenty or thirty thousands years — as Buttermilk Creek eroded back from the head of a hanging valley.

The park also offers a beach, cabins, fishing, hiking, deer bow-hunting, nature trails, pavilions, a playground, playing fields, recreation programs, and a campground with tent and trailer sites.