The municipality (originally the parish) is named after the old Nes farm (Old Norse: Nes—now the village of Nesbyen), since the first church was built there.

This represents a sandy peninsula or headlands (yellow triangle) formed by the confluence of two rivers: Rukkedøla and Hallingdalselva.

Reflecting this route, Hallingdal and its neighboring valley of Valdres were originally populated by migrants from Vestlandet and spoke a western dialect.

In recognition of this, Cardinal Nicholas Breakespear, who was in Scandinavia as papal legate in 1153, included these two valleys in the diocese of Stavanger.

It is bordered on the north by the municipality of Gol, on the east by Sør-Aurdal, on the southeast by Flå, on the southwest by Nore og Uvdal, and to the west by Ål.

The majority of the residents live in the villages of Nesbyen, Espeset, Eidal, Sjong, Børtnes, Bromma, Svenkerud, and Liodden.