Originally set in Ula, the county seat moved to Rosita in 1878, and to Silver Cliff in 1886 before settling in Westcliffe in 1928.

Thousands of men poured into the county during this time in the hunt for silver.

During the late 19th century a railroad line was connected through the Grape Creek Canyon but was permanently closed after a few disastrous floods.

After the mines were exhausted, the population dropped considerably and was replaced by cattle ranchers.

The county seat of Westcliffe is about 7,800 feet (2,400 m) and along with nearby town Silver Cliff lies in the Wet Mountain Valley which sits at the base of the Sangre de Cristo Mountains.

The peaks of the Sangre de Cristo Mountains to the west reach heights in excess of 14,000 feet (4,300 m) with Crestone Peak being the highest at 14,294 feet (4,357 m) and 7th highest fourteener in Colorado.

The only lake of size is the DeWeese Reservoir in the north end of the Wet Mountain Valley.

During the 2004 presidential election, well over 60% of Custer County voters voted for the Republican candidates including George W. Bush and Pete Coors.

Despite the surface appearance of a homogeneous political culture, there is considerable dispute among residents over planning.

One segment of residents (which includes ranchers) would like to see the County preserved in its present state as a mountain paradise with its rural ranching culture, with strict limitation on development.