The town was originally granted in 1761 by Governor Benning Wentworth to settlers from Colchester and East Haddam, Connecticut.

It was first settled in 1765; however, some grantees failed to comply with the charter, so Rumney was regranted to another group of settlers in 1767.

[1] The highest point in Rumney is 2,960 feet (902 m) above sea level on the southern slope of Carr Mountain.

The Baker River flows from west to east through the center of the town, roughly paralleled by New Hampshire Route 25.

Rumney village, the town's central settlement, lies just north of the Baker River.

Directly to the south, across the Baker River and adjacent to Route 25, is the village of Rumney Depot.

Rumney is home to Polar Caves Park, which is well known for its geological views and glowing rock formations.

These cliffs are mostly made of schist, with some being granite, and vary from long smooth slabs to wildly overhung roofs and aretes.