Founded in 1818, when Alabama was still a part of the Mississippi Territory, it is the oldest continuing library in the state.

It was first located in the office of attorney John Nelson Spotswood Jones, in the Boardman Building, which is now a part of Constitution Hall Park.

The area enjoyed rapid growth with the influx of government employees involved in the development of the space program, including the United States Army and NASA, and the library eventually needed more space as early as 1969.

The current facility's main branch, sometimes referred to as "Fort Book" for its fortress-like appearance, opened on Monroe Street in April 1987 and serves as the headquarters for the Huntsville Madison County Public Library System.

The building contains 123,000 square feet (11,400 m2), has a seating capacity of 930 and contains over 530,000 volumes, with administrative offices located on the third floor.