Most of the mountainous district is sparsely populated, and is part of the Tanzawa-Ōyama Quasi-National Park.
Ashigarakami District was one of the ancient subdivisions of Sagami Province, per the Nara period Ritsuryō system, under the name as Ashinokami District (足上郡, Ashinokami-gun).
Following disasters caused by eruptions of nearby Mount Fuji, a portion also came to be held as tenryō territory administered by the Tokugawa shogunate.
After the Meiji Restoration, it initially formed part of the short-lived Ashigara Prefecture, before was established as a district of Kanagawa Prefecture under the cadastral reform of 1878.
On April 1, 1972, part of Ashigarakami District became the city of Minamiashigara.