Built in the 1960s by Australian architects Kevin Borland and Daryl Jackson, the Swimming Centre is considered to be a fine example of Brutalist architecture.

[2] It is named in honour of Prime Minister Harold Holt, the local member of parliament (representing the Division of Higgins) until his apparent drowning death while the facility was under construction.

[1][3] In 1966 the Malvern Council commissioned architects Kevin Borland and Daryl Jackson to design a new swimming complex, as the Municipal Baths had become declared 'inadequate and outmoded' by the early 1960s.

[2] In 1988 the centre underwent a renovation with the additions of a hydrotherapy pool, spa, sauna and multi-purpose room for aerobics and yoga classes.

[1][3] The Harold Holt swim centre is categorized as a brutalist building by Stuart Harrison in the journal Architect Victoria, Summer 2003.

He goes on to discuss the building as a merger between different architectural styles that create a factory like feel within a suburban area much like Edmond and Corrigan's 1978 St Joseph's Chapel, Box Hill, Victoria, Australia.

"[7] The Harold Holt Swimming Centre sits on a site of approximately 207 x 57 metres and consists of several pools with varying functionality and use.

[1] The complex also houses a hydrotherapy pool, spa, sauna and multi-purpose room for aerobics and yoga classes, which were additions in 1988.