The first settler on record in the area was William Lewis Connelly, a colonel in the North Carolina militia, for whom the town was named.

In 1838, Colonel Connelly built a log home and operated a way station where fresh horses could be made available for the stagecoach line that ran from Salisbury to Asheville.

[4] In 1885, the area was found to have a mineral spring, thought to be beneficial in healing a large number of diseases.

Mrs. Elmira Connelly opened her spring to others, and before long people were arriving by horse and wagon, and by train, to cart water away in five-gallon demijohns.

It is bordered to the west by the town of Rutherford College and to the north by Rhodhiss Lake on the Catawba River.

According to the United States Census Bureau, the town has a total area of 5.0 square miles (12.9 km2), all land.