[2] The park occupies a plot of 23 acres (9.3 ha) on the west side of New Bedford's southern peninsula, between Rodney French Boulevard and Brock Avenue.

It takes its name from the estate of Joseph Congdon, whose 1839-40 Gothic Revival house stands near the center of the park.

Congdon named his estate "Hazelwood", and built an English garden on the then-forested Clark's Point peninsula.

By the late 1890s, development had encroached on the area, and the city of New Bedford was purchasing land for use as public parks.

It hired Warren Manning, who had previously worked for Frederick Law Olmsted, to plan the park.