[12][15] Her husband's death had left Dolley Madison in a financially difficult position, so to reduce her expenses she took up residence in the house in November 1837.

[20] General George B. McClellan used the house as his Washington-based headquarters after the First Battle of Bull Run during the Civil War.

[4] The Cosmos Club immediately improved the height of the third floor by raising the roof, and added a large meeting hall by building a single-story 23 ft 8 in (7.21 m) extension to the south side of the house (with skylight).

The addition consisted of a ground floor with an entrance to the building on H Street NW, a cloakroom, and a connecting door to the assembly hall on the south side of the building; a second story with meeting rooms for the Cosmos Club as well as other societies which might use the premises; and a third story with lodgings and a meeting room for the Cosmos Club's board of directors.

25 Madison Place NW, the building immediately to the south of the Cutts–Madison House (against which its three-story assembly hall addition abutted).

[7] The Cosmos Club vacated the Cutts–Madison House in 1952 to move to new headquarters in the Townsend Mansion at 2121 Massachusetts Avenue NW, at which time the building was purchased by the U.S. government and used for offices.

[22] The Park Commission's charge was to reconcile competing visions for the development of Washington, D.C., and in particular the National Mall and adjacent areas.

[24] At nearby 1616 H Street NW, the Brookings Institution purchased the rear garden from the private owners of the Decatur House and built an eight-story Modernist office building there.

[25][26] Opposition to the demolition of the Cutts–Madison House and other buildings on Lafayette Square began forming shortly after the plan to raze the structures was announced.

Senators James E. Murray and Wayne Morse, several members of the House of Representatives, and citizens of the District of Columbia lobbied to defeat the legislation authorizing the demolition of the buildings.

"[28] The AIA established a committee to develop plans to save the buildings and adapt the new structures so that they incorporated the style and feel of the older homes.

[29] The newly elected Kennedy administration indicated on February 16, 1961, that it was anxious to retain the existing historic homes on Lafayette Square.

[31] In February 1962, First Lady Jacqueline Kennedy lobbied General Services Administration (GSA) director Bernard L. Boutin to stop the demolition and adopt a different design plan.

[34] Mrs. Kennedy enlisted architect John Carl Warnecke, a friend of her husband who happened to be in town that weekend,[33] to create a design that would incorporate the new buildings with the old.

[32][33][36] Warnecke's design was based on the architectural theory of contextualism, where modern buildings are harmonized with the urban forms usual to a traditional city.

[7] Warnecke's design for the National Courts Building was to create tall, flat structures in red brick which would serve as relatively unobtrusive backgrounds to the lighter-colored residential homes like the Cutts–Madison House.

Since the mid-19th century, witnesses have claimed to have seen the ghost of Dolley Madison rocking in a chair in the space where the porch on the west side of the house used to be, smiling at passersby.