Due to beach erosion and weather damage over the decades, the final structure was moved multiple times, including to Port St. Joe in 2014.

The shoals running out from the cape extended 3.5 or 4.3 nautical miles; 6.4 or 8.0 kilometres (4 or 5 mi) and made it dangerous for all vessels nearing the coast.

The site was "deemed to be entirely secure from overflow or inundation" by the collector of customs at Apalachicola, Florida, who selected it, with the assistance of "two of our most experienced pilots."

The light station sustained serious damage at the hands of Union troops aboard the USS Kingfisher that landed in 1862 during the Civil War.

In 1869 the beach in front of the lighthouse was reported to be washing away and would need protection against encroachments of the sea during heavy storms.

In 1877 Congress appropriated $2,000 for protecting the site after the Lighthouse Board had reported 2 years earlier "The base of the tower is very nearly at the same level as the sea, which is but little more than 150 feet distant, the shore being of shifting sand.

The following year the Board noted "No appropriation was made; the site remained unprotected and on July 3, 1882, the tower was overthrown and completely destroyed."

In 1887 the sea was reported again gradually cutting away the shore and during the year had washed away about one-third the distance to where the new tower had been built (300 feet or 100 m).

It was, therefore, recommended "that the tower and dwellings be taken down and removed to a point on the inside of the peninsula a little less than 1+1⁄2 miles [2.4 km], about northwest from its present position where there is a good site and 8+1⁄2 feet [2.6 m] of water, in St. Joseph Bay, within 400 or 500 yards (370 or 460 m) of it.

This location is such that the bearing of the San Blas Shoals will be the same as now, and the increase of 1½ miles in the distance from the shore will be of little importance so far as its value as a coast light is concerned.

The work of dismantling the skeleton iron tower was begun in February 1896 and carried on until April 30 of that year when it was stopped because the appropriation was exhausted.

This sum was appropriated on June 6, 1900, at which time the Board reported: "that the property and material stored at Black’s Island was being cared for by a watchman appointed for the purpose."

In 1903 the Board sought and obtained authority from Congress to use $7,000 of the $15,000 appropriated for moving the tower, to erect two keeper's dwellings at the old site.

[citation needed] In July 2014, the lighthouse and two keepers' quarters were moved from Cape San Blas to George Core Park in Port St. Joe.