It is named after Colonel James Swan, of Fife, Scotland, who purchased the island and some surrounding areas and organized their colonization in the eighteenth century.

The main industry is lobster fishing, which employs 40 people full-time and many others in part-time work.

[6] Swan's Island was first charted in 1606 by Samuel de Champlain's expedition, and at the time was used by local Indians as a seasonal hunting ground.

James Swan, a member of the Sons of Liberty and Boston Tea Party participant,[7] purchased the island in 1786.

In 1791, David Smith, an American Revolutionary War veteran, became the first settler of European extraction on the island.

The state-run ferry, which began service in 1960, helped make Swan's Island a popular summer tourism destination.

28.8% of all households were made up of individuals, and 14.4% had someone living alone who was 65 years of age or older.

The racial makeup of the town was 98.17% White, 0.31% African American, and 1.53% from two or more races.

A travel agency specializing in trails, describes attractions for the island that include six historic homes dating to the 1700s, a listing on the National Register of Historic Places, a lighthouse, nature features that include nesting bald eagles, waterfowl, as well as deer, and trails, hiking, and camping.