Situated on the steep land arising from the eastern bank of the Hudson River to the bluff on top of it, the park's principal feature is the Walled Garden, inspired by ancient Indo-Persian gardens, in which are found a small Grecian-style open-air amphitheater with two opposing sphinxes crouching atop paired Ionic columns; a classical pavilion; a stoa and loggias; and a circular, open-air tempietto called the Temple of the Sky.

[4][5] Politician Samuel J. Tilden a former governor of New York state and a failed Presidential candidate, rented the mansion from Waring in 1879, and bought it and the estate soon after.

In an article in The American Architect he complained about the "resulting incoherence [due to the fact that] lacking a tradition at home, our minds are furnished with pictures of every sort of formal garden."

[14] Bosworth himself preached that the topography of the land on which the garden was to be built, and the position of the house on the property, should determine what kind of features it should contain.

Instead, Untermyer's estate was rocky and irregular with hillocks, mounds, outcroppings and dips, and it sloped steeply down to the Hudson River.

This 3-acre (1.2 ha) "Indo-Persian kind of garden", as Bosworth called it, had crenellated of walls of beige bricks on three sides, with octagonal towers, all covered in cream-colored stucco scored with a diamond pattern.

[19] The Walled Garden also featured the Temple of the Sky, an open-air circular 30-foot (9.1 m) colonnade of 14 Corinthian columns topped by an entablature, overlooking a swimming pool.

[5] Because of inadequate funding, much of the property was not maintained; a number of structures gradually fell into disrepair, and parts of the site became overgrown, reverting to woodland.

In 1976, the Untermyer Park and Gardens experienced a period of vandalism and unrest, as neighbors reported hearing chanting, and seeing torches being carried through the woods at night.

[23] David Berkowitz, the notorious "Son of Sam" serial killer, reported being part of a Satanic cult that met in the grounds of the park.

In the 1970s an effort was made to restore the garden by Yonkers Mayor Angelo Martinelli, architect James Piccone and Larry Martin, but the campaign was short-lived and the property deteriorated again.

[1] In the 1990s community leaders such as Nortrud Spero and Joe Kozlowski persuaded Mayor Terence Zaleski to purchase more of the original estate's land with the help of the Open Space Institute resulting in the 43 acres (17 ha) of the park today.

[26] While conceptually Indo-Persian, many of the elements that Bosworth designed for the Walled Garden are derived from Greek models, such as the Temple of the Sky, a tempietto, on the western edge of the garden, a circle of Corinthian columns made of Alabama marble[28] topped by an entablature, with a large swimming pool with lion's heads sculpted by Frederick G.R Roth spouting water,[28][notes 1] the Doric stoa on the eastern wall for guests to relax in, and the open air amphitheater at the north end where performers would entertain guests.

[28] Other architectural elements, such as the great gate through which one enters the garden, and the tiled stage of the amphitheater, were based on Mycenaean motifs.

[29][26] The Overlook features two ancient Roman monolithic cipollino marble columns which came from the estate of noted architect Stanford White.

It was buried beneath vegetation and dirt for many years, until it was rediscovered and rebuilt using elements of the original, and connecting it to the watercourse of the Temple of Love, which includes waterfalls, ponds and bridges.

Actual graffiti inside the gatehouse was preserved, and the plantings of the garden are meant to give the feeling of being in an abandoned, ruined house.

[26][33] With its restored pergola and parterre layout, vegetables and fruit trees are planted in a formal configuration with an eye to aesthetic beauty.

The terrace below the garden will feature a row of bronze sculptures of Hudson River painters by the noted artist Greg Wyatt.