It became an encampment site and refuge for Native American migrant workers from across the North Pacific coast, many of whom were employed seasonally as hop pickers.

[1] Initially prohibited, the necessity of unloading ballast led to the designation of an offshore location south of Yesler Wharf as a dumping site.

[4] Beginning with Henry Yesler's sawmill in 1854, waterfront companies also began dumping waste material into the water, using the piled debris for land reclamation.

[6] The Oregon Improvement Company's (OIC) construction of two large docks near the site greatly accelerated this process, with dockworkers discarding hundreds of tons of ballast into the water below.

Workers would typically stay on the island for several days to share news and trade goods, before traveling by rail to inland hop farms during the summer harvests.

Before the mid-19th century, these northern nations had regularly conducted slave raids in the Puget Sound, but instead adopted migrant labor to fund potlatch ceremonies.

The 1889 Great Seattle Fire did not spread to Ballast Island, but destroyed the adjacent Oregon Improvement Company docks, later rebuilt at a larger scale.

After repeated pressure by the Seattle Chief of Police to remove the native residents, the OIC ordered their eviction in late January 1890, giving less than a week to leave the island.

Although many were still employed as migrant farm workers, native residents of Ballast Island increasingly turned to merchant work, selling seafood and crafts to tourists and marketgoers.

[17] On March 7, 1893, settler vigilantes torched and destroyed the West Seattle Duwamish village of t̕uʔəlalʔtxʷ, 'Herring's House',[18] forcing the inhabitants to flee to Ballast Island.

[21] Portions of the island were covered by an elevated extension of Railroad Avenue,[22] alongside dredging operations to accommodate a slipway between the OIC docks.

[29] Public art commemorating the Duwamish presence on Ballast Island was installed in November 2023 at the newly-opened Pioneer Square Habitat Beach, part of Waterfront Park.

Described by the Office of the Waterfront as a "interpretive viewing experience", the work consists of four stone columns, erected in a partnership between the Washington State Department of Transportation and various local tribal entities.