The first minister to hold this position was Senator Ian Campbell, who held it from 7 October 2003 until 18 July 2004.

On 18 July 2004, the position was transferred to Jim Lloyd, then Federal Member for Robertson.

When the Fourth Howard Ministry started on 22 October 2004, Lloyd retained the position, and did so until the 2007 Federal Election when Kevin Rudd and the Australian Labor Party won government.

Rudd's first ministry eliminated the position of Minister for Local Government, Territories and Roads.

The responsibilities were given to Anthony Albanese, Minister for Infrastructure, Transport, Regional Development and Local Government.