The same location is also shown in artworks by Joseph Lycett, Walter Preston and Edward Close which are particularly significant as they show Newcastle's Government House and gardens.

Convicts sent to Newcastle endured harsh living standards, intense manual labour in the coal mines and were under constant surveillance.

Furthermore, the association between the former military barracks buildings and the port initiative (the pier) was of significance to capital works programs of the Colonial Government, convictism and early technology in Australia.

A small rail system for haulage was used to take coal to the port directly downhill from the mine site to be loaded onto ships for export.

The Commandant had his residence in a prominent place at the top of this street where he was able to view what was happening in the settlement, including work at Collier's Point and Nobbys Island.

[8][1] In the 1940s Jonathon Dixon carried out research on the site and attempted to locate the position of the first convict coal shaft by surveying an early map.

[9] Dr B W Champion (1949) also supports Dixon's location of the convict coal mine, adding that it was sunk approximately 20 yards inside the Mental Hospital gates.

[13][1] Lachlan Macquarie on his tour to the northern settlements in 1821 stated in his journal that he stayed at Government House in Newcastle, finding it very comfortable.

[14] Macquarie made several visits to Newcastle, and in 1812 he stated "...immediately on my landing respecting the inspection of the settlement, I went with Mrs. M. & c. to view the coal mines...".

This was a major colonial public works project, undertaken to join the mainland with Nobbys Island and establish a safe port entrance to facilitate the coal export trade.

A convict chain gang in Newcastle was employed to build the foundations for the officers quarters and soldiers barracks in 1838, as well as to create the military parade ground.

[32][33] Newcastle was the fourth main government-run asylum to be opened in New South Wales, the others located at Tarban Creek, Parramatta, the first being at Castle Hill, which later become a gaol.

[36] When it was suggested for a second time, a Newcastle newspaper, the Chronicle stated that it was dangerous to have insane people living in such close proximity to residents.

[38] Manning's decision to separate the imbelices and idiots required extra space, at a time when overall patient numbers were increasing, due in part to the effects of the 1890s depression.

Manning had intended to have the younger patients housed on a separate site, but the government's funding was limited during the depression so additional wards were constructed.

A visit from the Deputy Inspector General of Mental Hospitals in 1918 found the newer wards of a high standard but determined that the older buildings were in poor condition.

There were numerous outbuildings to Government House and an artwork completed in 1820 (artist unknown) shows these as well as a path to the left of the building leading uphill.

[1] The front of the original building was demolished in the mid 1800s for the realignment of Church Street; however a significant portion of the parsonage was retained when additions were made to Reception House by Architect Walter Vernon.

[1] The landscape of the current hospital grounds shows an exposed quarried landform on the south side, approximately 20 metres high.

The exposed rock has eroded somewhat due to environmental conditions; however this landscape remains a noticeable and strong feature of the site, the quarried section that formed a wall runs the full length of the southern boundary.

[1] This park is situated directly across the road (Watt Street) from the current James Fletcher Hospital, on the east side of the precinct.

[1] The park is reasonably well-maintained by Newcastle City Council; much of the open space has remained undisturbed, with the exception of the garden beds on the west side (Watt Street), which were replaced in 2005.

[46][1] Newcastle Government House was listed on the New South Wales State Heritage Register on 22 March 2011 having satisfied the following criteria.

It meets this criterion of State significance because it demonstrates patterns of economic and social development of the early colonial period from government-controlled and convict-worked industry, to the arrival of free labour and the beginnings of private enterprise.

It demonstrates an important aspect of law and order through its history as a military barracks and as the source of supervision required for the prisoners who remained in Newcastle to complete the Macquarie Pier.

The site is also significant for its association with Captain George Barney, one of Australia's most important Colonial Engineers during the mid 19th Century, (whose works include the Victoria Barracks in Paddington and the design of Circular Quay) and with Dr Frederick Manning Norton, who made a considerable contribution to the welfare of the insane and the improvement of mental health care in NSW.

The site has a high degree of aesthetic significance for its position toward the top of the eastern side of the Hill area and has remained virtually intact since first dedicated.

Many individuals have found creative inspiration from its landscape since 1804 and their artistic works have documented the transformation of this unique place shaped by human intervention (e.g. Ferdinand Bauer and Joseph Lycett).

The place is an early example of a Colonial public works project (Military Barracks) as well as the natural landscape transformed by convict labour.

An important site of early interaction between Aboriginal and European people, the place may have potential to contribute new knowledge about the relationship between these two cultures in the colonial period.