General Adrian Northcutt, a prominent local farmer and military officer, was the prime advocate for the site's selection.

The congregation was formed in the 1890s, and a Mormon chapel was built in Northcutt Cove just north of Altamont in 1909.

The latter is a gorge sliced by the upper watershed of the Collins River (Altamont is part of this watershed) as the river descends from the plateau to the Highland Rim en route to its mouth at the Caney Fork in adjacent Warren County.

Altamont is surrounded by hills on all sides, and the central area of the town itself straddles a flattened-out hillslope.

Tennessee State Route 56 (Main Street) connects Altamont to Beersheba Springs and McMinnville to the north and Coalmont, Tracy City and Monteagle to the south.

Tennessee State Route 108, connects the town to Viola and McMinnville to the northwest and Whitwell to the southeast.

According to the United States Census Bureau, the town has a total area of 20.2 square miles (52.2 km2), all land.