The first white explorers to pass through the area were the expedition led by Major Thomas Mitchell in 1836 who spoke enthusiastically of the landscape's green hills, soft soils and flowery plains, describing it as ideal for farming and settlement, naming it Australia Felix.

[3] The first white settlers in the area were the Henty brothers who had landed in Portland, Victoria in 1834 and who claimed 28,000 hectares between what are now the towns of Casterton and Coleraine.

[6] By the 1890s, increasing soil erosion saw wheat-farming around Casterton begin to decline and it was largely replaced by meat, wool and dairy farming.

Casterton's population expanded in the early 20th century, especially in the 1920s with the arrival of large numbers of soldier-settler farmers and during the post-war era in the 1950s.

[10] Casterton lays claim to be the birthplace of the breed of working dog known as the kelpie, a Scottish term meaning 'Water Sprite' and a name given to a black and tan bitch British working collie owned by Scotsman George Patterson, a farmer who lived north of Casterton in the 1870s.

Encouraged by the success of this, the Scouts, to celebrate the 1941 opening of the town's new Scout-hall, carved the Fleur de Lys emblem into the hill and lit it up at night with the aid of a series of tins filled with oil-soaked rags which were set alight.

[24] The town has a public outdoor swimming pool, a hospital, a secondary college and State and Catholic primary schools.