The town is home to the Mount Washington Hotel at Bretton Woods and to the Highland Center at Crawford Notch, the Appalachian Mountain Club's four-season lodge.

[5] Although the surface is uneven and bounded by mountains, farmers found the soil "strong and deep".

By 1859, Carroll had a starch factory, and two lumber mills that produced boards, shingles, clapboards and laths.

Hannah and Abel Crawford turned their log cabin into an inn, then built the Notch House in 1828.

[7] George Van Dyke then leased land and the Zealand mill from Henry, and constructed the Little River Railroad in 1893.

New Hampshire Route 115 leads from Carroll to Jefferson, with scenic views of Whitefield, Lancaster and the northern White Mountains.

A substantial portion of the town is part of the White Mountain National Forest, including Cherry Mountain ("Mount Martha", at 3,554 feet (1,083 m) above sea level), which is traversed by the Cohos Trail; and part of the Dartmouth Range, which contains Mount Deception, the 3,670-foot (1,120 m) summit of which is the highest point in town.

The southeastern corner of town boasts the Mount Washington Hotel and the height of land in Crawford Notch.

The Ammonoosuc River rises at the confluence of several brooks and flows westward generally along Route 302 through town.

[1] Carroll lies almost fully within the Connecticut River watershed, leading south to Long Island Sound;[10] the southernmost tip of the town surrounding Saco Lake, however, is part of the Saco River watershed, leading east to the Gulf of Maine.