In 1826 a bushfire raged through the area destroying Milson's home, orchard and dairy and farm[8] which he subsequently rebuilt (refer to 1840s map).

[6] Milson lived in the vicinity of Jeffrey Street until 1831 when he built a new home, "Brisbane House", on his 50 acres facing Lavender Bay.

[12] The last of the family's holdings in the lower North Shore area were resumed in the early 1920s for the construction of the Harbour Bridge and associated roadways.

[13] After World War II, Milsons Point became largely residential, but its location made it a highly desirable area for development.

The suburb, in close proximity to the Sydney central business district (CBD), was a prime spot for growth and urbanization.

It fully embraces its status as a major residential and business area, with its proximity to the Harbour Bridge, making it one of the most sought-after locations in Sydney.

Today, the area is home to many contemporary apartment buildings and office complexes and remains a popular location for both locals and visitors in closeness to the Sydney Harbour.

It was closed by the state government in 1990, despite earlier promises by Premier Nick Greiner that it would remain open and a legal challenge by the school's parents.