Johan Jost Herchheimer, a farmer who also engaged in trade and transport on the Mohawk River, settled in the German Flatts District of Albany County in 1725, on the south side of the Mohawk River, within the present-day town of German Flatts, incorporated in 1788.

He owned 5,000 acres (20 km2) of the land, including a strategic portage around Little Falls on the Mohawk, where his eldest son Nicholas Herkimer established a farmstead.

In 1788, the town of Herkimer, on the north side of the river, was incorporated and named after him.

By 1770, Nicholas had established great wealth through farming and trade during the wars, and had also gained military experience as a captain of a militia.

In 1906, the murder trial of Chester Gillette brought many spectators to the county courthouse in Herkimer village.

According to the United States Census Bureau, the town has a total area of 32.2 square miles (83.5 km2).

[3] West Canada Creek empties into the Mohawk River by Herkimer village.