The square is named for James Madison, fourth president of the United States.

The oldest building on the square is the Sorrel–Weed House, at 6 West Harris Street, which dates to 1840.

[1] In the center of the square is the William Jasper Monument, an 1888 work by Alexander Doyle memorializing Sergeant William Jasper, a soldier in the siege of Savannah who, though mortally wounded, recovered his company's banner.

[3] Madison Square features a vintage cannon from the Savannah Armory.

[2][5] The square also includes a monument marking the center of the British resistance during the siege.