The city is named after Golconda in Hyderabad, India, famed in the 19th century for its diamond mines.

In 1840, the Buel House, a single-family home presently-owned by the Illinois Historic Preservation Agency, was built.

Some 13,000 Cherokees, led by Chief Bear Paw crossed the Ohio River at Golconda by ferry as part of the infamous "Trail of Tears" to Oklahoma.

Because of the threat of disease, the Native Americans were not allowed to go into any towns or villages along the way; often this meant traveling much farther to go around them.

They were not allowed passage until the ferry had serviced all others wishing to cross and were forced to take shelter under "Mantle Rock", a bluff on the Kentucky side, until "Mr. Carpenter had nothing better to do".

Many of the Cherokee were rescued and sheltered by the Carpenter family, one of the founding leaders of Golconda.

Tommy Lee Jones in "U.S. Marshalls" mentioned Golconda, among others, when telling Law Enforcement where to set up road blocks along the Ohio River.