Situated on the northwestern reaches of the Kurnell Peninsula and linked with the sandstone of Sutherland Point in the east, the beach is characterised by silver-coloured sands, hence the name, and fourteen rockwall groynes which project into Botany Bay.

The eastern point of the beach is the site where Captain James Cook first set foot on Australian soil in 1770, which marked the beginning of Britain's interest in Australia and in the eventual colonisation of this new "southern continent".

As Cook and his party landed, a spear was thrown by one of the warriors before they retreated and commenced to ignore the intruders for the entire time the British were anchored in the bay.

Nonetheless, about 18 years later, when Captain Arthur Phillip and the First Fleet arrived in early 1788 to establish an outpost and penal colony, they found that the Bay and its surrounds did not live up to the promising picture that had been painted, as the area had poor soil, no secure anchorage, no adequate supply of fresh water, and the vegetation was too dense.

To cope with the area's increasing visitation Holt built the first wharf at Kurnell just adjacent to the Obelisk and a steam ferry began to operate sometime around 1882.

The rocks at the eastern tip of Silver Beach showcase the beginning of the large sandstone Kurnell Peninsula and its 8 km (5.0 mi) of rocky shore located in Botany Bay National Park.