Charles Carroll of Carrollton, a Roman Catholic lay leader, wealthiest man in America at that time and the only surviving signer of the Declaration of Independence, laid its cornerstone on July 4, 1828.

[12] The tower remained the tallest structure in the United States until 1846, when Trinity Church, New York on Wall Street was erected, and the tallest in Baltimore until the completion of the spire of the First Presbyterian Church at West Madison Street and Park Avenue in the Mount Vernon-Belvedere neighborhood in 1875.

[13] In 1921 the tower was purchased for $14,500 by the Union Oil Company, which planned to tear it down and put a gas station in its place.

After strong objections by the community, by 1928 enough money had been raised to purchase the tower and present it to the City of Baltimore as one of its first preserved local historic landmarks.

In 1985, other historic sites and homes were added to the newly created Baltimore City Life Museums system.