Specifically, the fort is located on the site of earthworks originally built to defend New York City during the American Revolution.

General Israel Putnam constructed the first earthen fortification on this site starting in April 1776, and armed it with eight cannons for the defense of New York Harbor.

The fort was reconstructed as a square with four corner bastions, and was named after the Federalist New York governor John Jay.

With the election of Thomas Jefferson as President in 1800 there was a shift of power from the Federalists, of which Jay was a prominent member, to the Democratic-Republican Party.

This system of coastal fortifications is credited with discouraging the British from taking any naval action against the city during the War of 1812, who preferred easier targets in the Great Lakes, the Chesapeake Bay (resulting in the burning of Washington, D.C.), and the Gulf of Mexico below New Orleans.

The Greek Revival style barracks, unified by two-story Tuscan porticos first served as officers' and enlisted men's housing for the permanent garrison.

That same year the Ordnance Department established the New York Arsenal as a separate installation, adjacent to but not part of Fort Columbus, as a major depot taking delivery of contracted manufactured arms and weapons and distributing both contract and federally manufactured weapons to army posts across the nation.

In 1836, the South Battery became the Army School of Music Practice, training young boys to become company drummers and fife players and regimental musicians.

Outgoing President James Buchanan initiated the first effort, but a battery garrisoned by cadets from The Citadel, The Military College of South Carolina fired on the Army-chartered New York-based steamship Star of the West on January 9, 1861 as it entered Charleston Harbor.

Major General William H. C. Whiting (CSA) died of dysentery in February 1865 in the post hospital shortly after his surrender at the Battle of Fort Fisher, North Carolina.

[6] At the turn of the century, Fort Columbus and Governors Island began to draw the attention of President Theodore Roosevelt's Secretary of War, Elihu Root, a former New York City lawyer.

Root sought to enhance the island army post to better serve the army's needs, to fend off efforts by the City of New York to close the post and reclaim the island as a city park and a rarely stated need to provide a quick means of federal protection of Wall Street, the Customs House and Sub-Treasury in Lower Manhattan.

The retention of the fortifications indicated Root's interest in retaining the historical structures, as some of the army decision makers he supervised were trying to accomplish their demolition.

In one of his departing acts as Secretary of War in February 1904, Root restored the original name of Fort Jay to the fortification and to the army post that had evolved around it.

During World War II Fort Jay was the headquarters of First Army in the early part of the war, and later the Eastern Defense Command (EDC), responsible for all Army units and defense coordination in the northeastern United States, and in the east coast states from Maine through Florida.

[7] In November 1964 after a year of study to identify ways to downsize Department of Defense installations, the U.S. Army announced the closure of Fort Jay.

On January 19, 2001, Fort Jay, Castle Williams and a surrounding 23 acres were proclaimed part of the Governors Island National Monument, administered by the National Park Service with Fort Jay recognized as being one of the finest remaining examples of the Second System of American military fortifications.