The island is located to the east of the center of the river channel, 84 miles (135 km) north of the river's mouth at New York City, roughly 1,200 feet (370 m) offshore, southwest of Norrie Point in Staatsburg, and opposite the mouth of Black Creek in the town of Esopus on the west shore.

[1] It is part of Margaret Lewis Norrie State Park and of the Hudson River Watertrail and has campsites, picnic areas, trails, and fishing access points, but can be reached only by boat.

[1][4][6] The Lenape Indians are presumed to have made use of the island, and a stone on the east shore shows signs of human working, resembling a megalith.

[7] In October 1777, during the Revolutionary War, a British fleet laid off Esopus Island prior to destroying Kingston, then the provincial capital.

Friends had given him money to buy a tent, a canoe, and stores for his retreat to the island, but instead of food he bought the paint, brushes, and rope for rappeling, saying that he would be "fed by ravens".