The name Kharmang was given to the valley in the era of Ali Sher Khan Anchan, who built many forts in this region because of its strategic importance.

According to the Gazetteer of Kashmir and Ladak (1890), Kharmang is an old ilaqa of Baltistan that consists of the right bank of the Indus from the border of Ladakh to the village of Pari, and from the source of the Shingo River to the village of Tolti on the left bank of the Indus.

The main source of income is subsistence livestock rearing for their livelihood and agriculture and unskilled man power working abroad and urban centers of Pakistan.

Many Kharmang people work in Gulf countries with a majority in Kuwait, Bahrain and Saudi Arabia.

Natives of Kharmang Valley speak Balti, Shina and predominantly follow Shia Islam, with the exception of a few people who belong to Noorbakhshi who live around Mehdi Abad.