Its west side is characterized by a steep, rocky rise of about 40 feet (12 m) above an otherwise relatively flat landscape.

The traditional main gate is located at the junction of Washington and Centre streets, with a wall section that includes a built-in structure originally used as a covered shelter for a streetcar stop.

The park was established by a donation from Armenia White, a local philanthropist, and designed by Charles Eliot.

A number of the plantings lining the park's Washington Street boundary date to the initial period of development, and include several specimen trees and shrubs.

The park's recreational facilities were expanded in the 1930s with funding from the Works Progress Administration, at which time the smaller of the two ponds was filled in, and replaced by a swimming pool.