The mountain is located in Pittsfield State Forest and is traversed by the 12.1 mi (19.5 km) multi-use Taconic Skyline Trail.

The summit is known for its extensive stand of wild azalea and is wooded with northern hardwood tree species.

Most of Smith Mountain is located within Hancock with lower east slopes within Pittsfield.

The west side of the mountain drains into Wyomanock Creek, then into Kinderhook Creek, thence into the Hudson River and Long Island Sound.

The east side drains into Smith Brook, thence to the Housatonic River and Long Island Sound.