The station was opened as "Harrow" on 2 August 1880, when the Metropolitan Railway was extended from its previous terminus at Willesden Green.

Like some other Underground stations, the name is an example of marketing rather than precision; in this case the town "proper" of the same name is at the top of Harrow Hill (i.e. Harrow-on-the-Hill), while the station is located at the foot of the hill to the north, which at the time of opening was a small hamlet called Greenhill and has since become the main town proper of Harrow.

Had the governors of Harrow School not made objections during the planning stage, it is possible that the Metropolitan Railway might have followed a different route taking it closer to the town centre on the hill.

The GCR ran on the former Great Central Main Line, an intercity trunk route and provided services from Harrow to destinations such as Rugby, Leicester, Nottingham and Manchester.

Chiltern Railways started operating in 1996 after the privatisation of British Rail and typically provide two trains per hour between Aylesbury and London via Harrow.

An office block called Avanta House was built in the 1980s on top of the station's College Road entrance.

[11] Harrow-on-the-Hill gained step-free access from College Road as part of a tranche of access improvements scheduled for completion in 2020;[12] that work, which also introduced lifts to the platforms served by Chiltern Railways, was completed in March 2022 but passengers approaching the station from the south (Lowlands Road) are still required to ascend flights of steps.