The area at that time was also strewn with discarded oyster shells from many long years of gathering by the local aboriginal women.

[4] In the early 1790s, the Aboriginal man Woollarawarre Bennelong— employed as a cultural interlocutor by the British—persuaded Governor of New South Wales, Arthur Phillip, to build a brick hut for him on the point, giving it its colonial name.

[5] In December 1798, a half-moon battery was constructed at the extreme northern end of the Point, mounted with guns from HMS Supply.

The entire area was leveled to create a low platform and to provide suitable stone for the construction of Fort Macquarie.

[5] The existence of the original tidal island and its rubble fill were largely forgotten until the late 1950s when both were rediscovered during the excavations related to the construction of the Sydney Opera House.