Convict David O'Connor was hanged on the site in 1790 and it is said that his ghost haunts the Customs House to this day, offering people rum.

[6][1] The driving force behind the construction of the later sandstone edifice on Circular Quay was Colonel John George Nathaniel Gibbes, the Collector of Customs for New South Wales for a record term of 25 years from 1834 to 1859.

Colonel Gibbes persuaded the Governor of New South Wales, Sir George Gipps, to begin construction of the Customs House in 1844 in response to Sydney's growing volume of maritime trade.

The building project also doubled as an unemployment relief measure for stonemasons and laborers during an economic depression which was afflicting the colony at the time.

Various additions were made over the next century, particularly during the period of World War I, but some significant vestiges of the original Gibbes-Lewis building remain.

A further round of refurbishment was completed in 1999,[9] converting the building into the "tourism gateway" to Sydney ahead of the 2000 Summer Olympics.

The two-storey Georgian structure was designed by Mortimer Lewis and featured thirteen large and expensive windows in the facade to afford a clear view of shipping activity in Sydney Cove.

Floors in the perimeter building are generally suspended timber, joisted with added steel beams, while the framed core has reinforced concrete two-way slabs.

[2] The Sydney Customs House occupies a unique symbolic and physical position on the site of the First Fleet Landing.

The Customs House contains parts of the oldest surviving building of its type in Australia, used continuously for 145 years.

The Customs House embodies the work of three successive and individually distinguished government architects: Mortimer Lewis, James Barnet and Walter Liberty Vernon.

[11][1] Customs House, Sydney was listed on the New South Wales State Heritage Register on 2 April 1999 having satisfied the following criteria.

The Sydney Customs House occupies a unique symbolic and physical position on the site of the First Fleet landing.

[11][1] The place is important in demonstrating aesthetic characteristics and/or a high degree of creative or technical achievement in New South Wales.

The Custom's House embodies the work of three successive and individually distinguished official architects of New South Wales.

Although Walter Liberty Vernon and James Barnet greatly altered the work of Mortimer Lewis, they did so using similar external materials and proportions so as to generate an overall unity of construction.

[13][1] The place has potential to yield information that will contribute to an understanding of the cultural or natural history of New South Wales.

[12][1] The place possesses uncommon, rare or endangered aspects of the cultural or natural history of New South Wales.