In the period of the Revolutionary War it was among the most notable mansion of Newburyport, owned by the Tracys, the leading family of the times.
[2] His main interests outside of business, however, were St. Paul's Church, where he was a vestryman, and building the future City of Newburyport.
In 1775 Nathaniel had constructed and outfitted the first Yankee privateer, and was so successful that a fleet of warships followed, all acting under letters of marque.
For their wedding gift Patrick had constructed a new mansion on State Street (the center of the city), now the library.
The remaining vessels were detained in whatever port they were using for supply, the crews were imprisoned or impressed into the British Navy and the ships, unable to sail, were abandoned.
Following George Washington's policy of avoiding foreign wars and entanglements, Congress paid a significant portion of the national budget to corsair states such as Tripoli, but to no avail.
These events in the rise of American sea power were far too late for Nathaniel Tracy: "... he found himself hopelessly involved in financial difficulties, owing large sums of money which he could not pay, and with the close of the War, his wealth vanished like smoke....
His eleven children and those of his brother and sister went on to found families from which many leading men and women of New England were to descend, including Supreme Court Justice Oliver Wendell Holmes.
The home in Newburyport did not then pass out of the family; it was taken over by Nathaniel's brother-in-law and former business partner, Jonathan Jackson, entering a new period of splendor.
[5] The demand for books was so great that the directors began almost immediately seeking larger, better quarters, which would require more money.
[6] The means evaded them until 1860 when the Newburyport Lyceum agreed to sell tickets to lectures by persons of special invitation, who would do it for free, and donate the proceeds to the Library Building Fund.
[8] At this point Edward S. Moseley, a banker and one of the wealthiest men in Newburyport, placed himself and his friends on the subscription list.
The names are familiar from today's streets and structures: the Hales, Cushings, Currier, Coffin, Forbes, Moulton, and so on, and others from as far away as New York and London.
The Tracy home was totally redesigned, internally, externally and grounds, by Arthur Gilman of Boston, a native of Newburyport, "making no charge for his services.
[11] The final deed of conveyance specified among other conditions that the building must be used "exclusively for the city library," and that it "shall not be open for public use on the Lord's Day."