Priority was given to establishing the colony's first structures, and the settlers' interests were initially geared more towards temporary housing and a ready supply of freshwater (via the Tank Stream) than in conquering challenging topography.

One profitable industry that exploited local resources was the production of stone for the construction of housing and services in early Sydney: sections of Millers Point were known as "The Quarries", near Kent and the western end of Windmill Streets.

[1] The location of Millers Point, with its relationship to the waterfront, was ideally suited for shipping purposes, and merchants tapped into its potential by erecting private jetties, wharves, and storage for goods.

The village of Millers Point became a definitive one in the early 1830s, as maritime and other related enterprises began to radiate outwards from Sydney Cove, bringing with it residential and commercial facilities.

Valuable goods such as wool had to be loaded and unloaded at a rapid rate of turnover, with laborers required to be on call and, as such, in the nearby vicinity to respond to erratic shipping arrivals and departures.

These key characteristics of Millers Point distinguished it from other areas, and its unusual composition was reflected by the high level of rental housing, which in most other suburbs was an indicator of poverty and unskilled workforces.

The latter became particularly associated with the Dawes Battery military garrison but also served as a base for school and moral education and a forum for community gatherings in accordance with the accepted role of churches in the colony.

Certainly by the midpoint of the nineteenth century, a gradual overlaying of cultural features had evolved into a flourishing and distinct community, with various church denominations, a wide range of commercial and social services, and in 1850, the Fort Street Model School was opened, having been the original military hospital constructed in 1815 and renovated to architect Mortimer Lewis' design in 1849.

At the turn of the twentieth century the construction of Steven's tenement building- the first Sydney example of purpose-built brick flat development, using a French architectural concept- heralded a change in outlook brought about by the economic slump and consequent financial hardship on the part of Millers Point residents.

The reliance on shift work and seasonal employment meant that in periods of economic depression a considerable proportion of laborers in the area were unable to earn regular wages; the conflict over unionism and the Great Maritime Strike of 1890 combined to leave locals destitute or at best living on reduced incomes.

Although alerted to the presence of plague in other Australian ports, ship and wharf-owners opted to minimize the risk to the industry and their profits by disposing of dead rats found on the wharves into the harbor.

[1] Wholesale resumption of large portions of the foreshore and Millers Point was heavily criticized, with detractors citing the plague as a convenient excuse to allow the government to "seiz[e] a political opportunity."

The Sydney Harbour Trust, established in October 1900, was intended to modernize the commercial waterfront and held responsibility for the administration of wharf facilities as well as control of housing in resumed areas.

This was magnified through close social interaction within the area, irregular employment conditions, and the psychological sense of division created by the Sydney Harbour Bridge, tram, and railway tracks.

The relatively uniform socio-economic status of the Millers Point population was comparatively undiluted, but as the prospect of homeownership and suburban life became a viable goal, housing pressures eased.

Improved communication and transport technology-enabled laborers to live outside Millers Point and commute between work and home - such a dramatic lifestyle change also had a temporary impact on social cohesion.

In the same year, the Millers Point precinct saw the relocation of architect Mortimer Lewis' home Richmond Villa, moved from the Domain to a government-owned site in Kent Street, to feature as the base for the Society of Australian Genealogists.

[1] In the early 1980s, the transfer of the Maritime Service Board's non-port related property to the portfolio of the Department of Housing commenced, with some Millers Point residents unsure as to the benevolence of their new landlord.

[1] Political fluctuations hampered the process of protecting the precinct, but in 1988 the New South Wales Heritage Council acknowledged the Millers Point Conservation Area as of state and national significance.

Both individual and group listings of buildings and structures relating to Millers Point have also been identified by the Register of the National Estate, including the Walsh Bay Wharves and the Rocks Conservation Area.

Although both are predominantly residential in character, the built environment of Dawes Point Quarter tends to contain larger houses, longer streets, the skyline presence of the Harbour Bridge, and broader views across the inner harbor, while the Millers Point Quarter tends to contain smaller houses, shorter streets, the greenery of Observatory Park on the heights and the skyline presence of city skyscrapers, and restricted views into Darling Harbour and Walsh Bay.

[1] As at 1 December 2003, much of Millers Point retained high archaeological potential, as demonstrated in reports by Higginbotham et al., notably Observatory Hill, Fort Street School, and its immediate environment, and under all c. 1900 buildings, external spaces and asphalted areas.

[1] Millers Point & Dawes Point Village Precinct has evolved in response to both the physical characteristics of its peninsular location, and to the broader historical patterns and processes that have shaped the development of New South Wales since the 1780s, including the British invasion of the continent; cross-cultural relations; convictism; the defence of Sydney; the spread of maritime industries such as fishing and boat building; transporting and storing goods for export and import; immigration and emigration; astronomical and scientific achievements; small scale manufacturing; wind and gas-generated energy production; the growth of controlled and market economies; contested waterfront work practises; the growth of trade unionism; the development of the state's oldest local government authority the City of Sydney; the development of public health, town planning and heritage conservation as roles for colonial and state government; the provision of religious and spiritual guidance; as inspiration for creative and artistic endeavour; and the evolution and regeneration of locally-distinctive and self-sustaining communities.

[1] Millers Point & Dawes Point Village Precinct has evolved in response to both the physical characteristics of its peninsular location, and to the broader historical patterns and processes that have shaped the development of New South Wales since the 1780s, including the British invasion of the continent; cross-cultural relations; convictism; the defence of Sydney; the spread of maritime industries such as fishing and boat building; transporting and storing goods for export and import; immigration and emigration; astronomical and scientific achievements; small scale manufacturing; wind and gas-generated energy production; the growth of controlled and market economies; contested waterfront work practises; the growth of trade unionism; the development of the state's oldest local government authority the City of Sydney; the development of public health, town planning and heritage conservation as roles for colonial and state government; the provision of religious and spiritual guidance; as inspiration for creative and artistic endeavour; and the evolution and regeneration of locally-distinctive and self-sustaining communities.

These include the Cadigal people of the area; Colbee, a Cadigal "leading man" in the 1790s; Lt William Dawes, first colonial astronomer (commemorated in the place-name Dawes Point); Jack "the miller" Leighton, wind mill owner; William Walker, merchant; Henry Moore, merchant; Robert Towns, merchant; Norman Selfe, engineer; Sisters of St Joseph, Catholic nuns at St Brigit's; the "Millers Point Push", gangsters of the Point; Ted Brady, wharf laborer, ALP, and Communist Part stalwart; Arthur Payne, first sufferer of the Plague in 1900; William Morris Hughes, union leader and later prime minister; RRP Hickson, chairman Sydney Harbour Trust; Waterside Workers Federation (WWF), union established in 1902; Jim Healy, general secretary WWF 1937–1961; Harry Jensen, Lord Mayor of Sydney 1957–1965; and the multi-generational "Pointer" families that give the Precinct its distinctive social character.

Millers Point & Dawes Point Village Precinct is of state significance for its landmark qualities as a terraced sandstone peninsula providing an eastern "wall" to the inner harbor and supporting the fortress-like southern approaches to the Sydney Harbour Bridge; for its aesthetic distinctiveness as a walking-scale, low-rise, village-like harbourside district with its central "green" in Argyle Place, and its vistas and glimpses of the harbor along its streets and over rooftops, the sounds of boats, ships and wharf work, and the smells of the sea and harbor waters; as well as for the technical innovations evident in the remolding of the natural peninsular landform from the hand-picked Argyle Cut to the ongoing leveling and terracing of the western slopes to the highly planned and mechanically created Walsh Bay and Darling Harbour docklands of the 20th century.

[1] The whole precinct demonstrates a range of technologies and accomplishments dating from the period the 1820s to the 1930s; this relates to landscaping, residential dwellings, industrialization, public areas, warehousing, maritime and religious structures.

It has a range of architectural styles that are both intact and excellent examples of their type, many of which are rare surviving shops and dwellings, with specific importance attributed to the Observatory, Fort Street School, and Holy Trinity Church, as well as colonial housing, hotels, and commercial amenities.

Finally, it has a range of early buildings with specific functions that are rare within the Australian context, such as the Lord Nelson Hotel and the Observatory[1] The place is important in demonstrating the principal characteristics of a class of cultural or natural places/environments in New South Wales.

Millers Point & Dawes Point Village Precinct is of state significance for its ability to demonstrate the principal characteristics of 19th and 20th-century Australian maritime harbourside or dockland precincts, such as close proximity between workplace and work residence; the development of new methods for moving produce and passengers between land and water; the interaction between natural elements such as water and wind and cultural elements such as wharves, boatyards and warehouses; and the constant remaking of the shoreline and its hinterland in response to changing economic, social, political and environmental factors in order for it to remain viable as a living, working place.