The parish was anciently called Hildersley, from Hildeslei in the Domesday Book,[2] and is recorded in a medieval inscription in the church.

Ilsley has been attributed by antiquaries as a leading contender for the uncertain site of the Battle of Ashdown (Alfred the Great's victory against the Danes).

[3][4] Shepherds and drovers would drive their sheep to the village, and stay overnight in one of at least 9 public houses until the fair in the morning.

[6] Its real property, farms and homes, was worth £4,490 (equivalent to £542,653 in 2023) and its population in the United Kingdom Census 1871 was 746.

[4] The parish Church of St Mary is partly Norman; has an early English style chancel and has an embattled tower; it was enlarged and repaired in 1845 and contains an old monument of one of the Hildesleys, the ancient lords of the manor.

At Keats Gore, a mansion near East Ilsley, the celebrated racehorse Eclipse was foaled in 1764, and afterwards trained.