This property located immediately west of Charlottetown's original "500 lots" was roughly eight times larger than the thirty-six 12-acre (4.9 ha) "estates" established in the northern part of the Queens Royalty.

In 1820, three commissioners reported to Governor Charles Douglass Smith their calculations of magnetic declination and placed stone markers to this effect in a field cleared immediately north of the Prince Edward Battery.

British military forces left Canada in 1905 and in that year the 16 acres (6.5 ha) property containing the Prince Edward Battery and a field adjacent to Government House were given to the City of Charlottetown to add to Victoria Park.

Following its transfer of ownership in 1873, the City of Charlottetown began making improvements to Victoria Park, including planting trees, removing stumps, constructing bath houses, and dredging Dead Man's Pond.

In 1903 the western part of the Victoria Park Roadway was constructed from the Prince Edward Battery west toward Duchess Point and then north along the shoreline to Brighton Road.