Originally known as Wheeler's Station, the town was renamed in honor of Judge William Carey, a prominent local landowner, in 1866.

[5] Caryville is situated in a valley between Cross Mountain to the west and a series of rugged hills to the east.

Cove Lake State Park lies immediately north of the town, and includes an artificial lake created by the impoundment of Cove Creek by Caryville Dam.

[6] As of the 2020 United States census, there were 2,212 people, 925 households, and 653 families residing in the town.

Caryville's economy consists mostly of restaurants, hotels and motels along the town's two exits (134 and 141) of Interstate 75.