It is located near South Africa's south coast, on the N2 highway, within the Hessequa region, 274 km east of Cape Town (about halfway between Cape Town and Knysna).

The historical Fourie House, dates back to the 1728, making it one of the oldest building in South Africa.

In 1716, Louis Fourie (1690s–1767) obtained grazing rights from Governor van der Stel and he settled alongside the Duivenhoks River.

In 1903 Heidelberg became part of the railway network and became an important transport link for the wool, wheat, fruit, and tobacco industries of the area.

The river, the Duivenhoks (Dovecote), was named by an explorer, Isaq Schrijver, who observed a lot of doves where the river flows into the Indian Ocean, at a place called Puntjie.