It is bounded on the north by Indianola, on the west by Suquamish, and on the south by Bainbridge Island.
Port Madison connects to Bainbridge Island via the Agate Pass Bridge to the southwest.
On Nov. 8 1824, John Work of the Hudson's Bay Company, while looking for potential sites for a trading post, recorded it as Soquamis Bay - a variation on the name of the Suquamish tribe which made its home on the western shore.
The Wilkes Expedition surveyed the bay on May 10, 1841 and named it for James Madison, the 4th president of the United States.
Today, Port Madison is a residential area and a popular destination for boaters.