In 1799 the second Governor, John Hunter, had the remains of Arthur Phillip's cottage cleared away, and a more permanent building erected on the same site.

The practice of "firestick" land management conducted by the aboriginal Darug tribe, which once dwelt in the area, is evident from certain scars to be seen on trees still standing (their bark being removed to build canoes).

These items include: buildings (such as Old Government House), relics (former observatory), historic plantings, archaeological sites (41 in all, including former roads, convict huts, stables, redoubt, lumberyard), vistas (across Parramatta and along George St to the former wharf) and natural items such as bushland.

[4] Within the boundary of the place, the layout of the major elements of the park retains much of the Governor Macquarie usage of the space.

[4] Old Government House at Parramatta was built by convicts and is the oldest surviving public building on the Australian mainland.

With its symmetrical proportions, shadow patterns from extended eaves and central portico it exhibits the 'Palladian' characteristics of Australian Old Colonial Georgian architecture.

[4] The entrance hall of Old Government House notably features a black-and-white chequered floor, in keeping with the tastes of the period.

However, due to the absence of building materials such as marble or granite in the early Sydney settlement, the floor had to made out of painted wood.

The Governor's dairy survives in its original setting, and has recently been stabilised and restored by the Park Trust.

The Macquarie stables and coachhouse were removed when the Great Western Railway line was pushed through the south-western section of the Domain.

In 1901 Gertrude and Lewis Taylor lived there with son Keith, born in the upstairs bedroom the following year.

[7][5] Governor Thomas Brisbane's Observatory site includes two transit stones, two marker trees (Himalayan or chir pine, Pinus roxburghii) to its south, two more chir pines near the Southern Domain gate house spaced the same distance apart as the two close to the observatory, centred exactly on the same north–south alignment extending through the gap in the transit stones (on the Great Western Highway) which probably mark the location of a marker stone), the Observatory Memorial (1880) obelisk and archaeological remains of the footings of both the 8.5-metre (28 ft) square observatory with its northern and southern domed ends and the former astronomer's cottage to its west.

[8][5] Other buildings include(d): On 2 April 1999, the property was listed on the New South Wales State Heritage Register as a site of State significance with the following citation:[5] This building, built c.1799 and enlarged 1815 to a design by Lieutenant John Watts, was associated with the administration of the colony from its early days until superseded by the 1845 Government House on the shores of Sydney Harbour.

[4] In July 2010, at the 34th session of the UNESCO World Heritage Committee, Old Government House and Domain, as well as ten other Australian sites with a significant association with convict transportation, were inscribed as a group on the World Heritage List as the Australian Convict Sites.

Of the 11 sites the Hyde Park Barracks, Old Great North Road and Cockatoo Island are also within the Sydney region.