Located along the banks of the Clinch River, Richlands began as a farming community and was named for its fertile "rich lands."

The Clinch Valley Coal & Iron Company began to develop Richlands in 1890, and company officials hoped Richlands' readily available coal, iron, and timber might make it the "Pittsburgh of the South."

[5] In February 1893, Richlands was the site of a mass lynching of five black railroad workers after it was alleged some of the men had robbed and beaten a white man.

Four of the railroad workers were arrested and held in the Richlands jail, but the jailor was overpowered by a mob of white townspeople, including James Hurt, a magistrate and member of the Richlands town council, and James Crabtree, a prominent businessman, who removed the four men and hanged them from the same tree.

Cedar Bluff borders Richlands to the east, and the community of Raven lies just to the west.

According to the United States Census Bureau, the town has a total area of 2.7 square miles (6.9 km2), all land.

[11][12] Virginian singer and pianist Bruce Hornsby wrote a song set in Richlands, entitled "The Road Not Taken".