Designed as part of the initial phase of the Edgbaston campus by architects Aston Webb and Ingress Bell, the tower was constructed between 1900–1908, and stood at the centre of a semicircle of matching red brick buildings.

On 1 October 1905, the Birmingham Post reported that Chamberlain had announced to the University Council an anonymous gift of £50,000 (the donor in fact was Sir Charles Holcroft).

This anonymous gift was announced some two months later in the Birmingham Post as "to be intended for the erection of a tower in connection with the new buildings at Bournbrook at a cost estimated by the architects at £25,000.

The original design for Old Joe is thought to have been based upon St Mark's Campanile in Venice, the latter having served as the inspiration for Sather Tower at University of California, Berkeley.

David Lodge's novel Changing Places tells the story of exchange of professors between the universities of Rummidge and Euphoric State, Plotinus (thinly disguised fictional versions of Birmingham and Berkeley), which in the book both have replicas of the Leaning Tower of Pisa on campus.