The area developed to serve the needs of many sportsmen after the middle of the 19th century, and was known as the "Inlet on Fourth Lake".

Inlet is surrounded by over one million acres (400,000 ha) of lands within the Adirondack Park that are easily accessible.

An American Tragedy, Theodore Dreiser's award-winning novel based on the murder of Grace Brown, included a scene at Arrowhead.

The actual murder took place in Punkey Bay on Big Moose Lake in 1906.

The book was later made into the movie "A Place in the Sun" starring Montgomery Clift and Elizabeth Taylor.

New York State Route 28 is an east–west highway across Inlet and is the only significant road in the town.

Inlet forms its western border with the towns of Ohio and Webb in Herkimer County.