[2]Holt House is intimately linked to the early history of both Washington and the nation through its association with such prominent figures as Thomas Johnson (first governor of Maryland), Thomas Jefferson, Benjamin Stoddert (first Secretary of the Navy), Benjamin Mackall (Georgetown merchant), John Adams, John Quincy Adams, and Andrew Jackson.

Perhaps to help increase his cash flow, he constructed a flour mill on the land surrounding Holt House.

When the first Quaker Meeting was held in Georgetown in 1806, the Shoemakers may have made the acquaintance of architect William Thornton then, but sold the property only three years later.

Additionally, his Quaker beliefs would suggest a more modest lifestyle, as he relied on his family and not employees or slaves to run the mills, and would not have allowed for managing a house of this size as well.

Beginning in 1812, George Johnson began borrowing large sums of money in order to rebuild the mills after they burned, employing a millwright to build "the best mill possible,"[11] and possibly may have used some of this money to construct a new house, or enlarge an existing house on the property as well.

Perhaps the first reference to a "house" on the property appears in an 1818 letter from George Johnson to his father-in-law, James Dunlop.

Roger also asks James Dunlop to assume payment on half of George's debts, as he considered him partially responsible as the under signer of the loans.

But, according to Smithsonian records, a month before this letter was sent, Roger had already transferred the deed for the mill to the Bank of Columbia, with George's father-in-law, James Dunlop serving as trustee "to set up and sell at public sale for cash or upon credit" against George's debts, which totaled $28,161.55 with interest.

In that same year, George once again approached Adams for assistance, soliciting a place as a Clerk in one of the Departments, to which John Quincy assured him he would "in no case recommend him.

It was not until 1835 that they sold the property to Dr. Ashton Alexander, a prominent physician from Baltimore, for whose family Alexandria, Virginia is named.

Dr. Alexander never resided in Washington himself, and in 1838, rented the house to Amos Kendall, postmaster general of the United States, a close confidant of Andrew Jackson, and one of the founders of the modern Democratic Party.

And for the whole three years not a dollar, so far, has been received for damages or rent.” Dr. Henry Holt, a former U.S. Army assistant surgeon from Oswego County, New York, purchased the property in December 1844.

By the time the Zoo purchased the property in 1889, Holt House was very dilapidated and badly in need of extensive repair.

Despite its structural problems, however, the building has retained its distinctive character of a nineteenth-century country villa, which appeals to visitors.

Congress has limited spending on the house: None of these funds in this or any other Act may be used for the Holt House located at the National Zoological Park in Washington, DC, unless identified as repairs to minimize water damage, monitor structure movement, or provide interim structural support.

[18]Holt House is one of the few remaining "suburban villas" in Washington, D.C, a design concept by the 16th-century Italian architect, Andrea Palladio.

The ground level of Holt House is thought originally to have had at least five rooms, less finely finished than the main floor and not a full story in height.

It may have been used as a residence for earlier owners of the mill, or used as the main family dwelling as the other parts of the house were completed, as was the custom in the 18th century.

Holt House was built of load bearing masonry walls, primarily of brick but intermingled with local stones and rocks.

The central block and matching wings on Holt House have gabled roofs with pedimented façades that are articulated with equally spaced modillions.

The main level entrances on both the north and south sides of the central block were three panel double-doors with glazed sidelight sashes.

A significant architectural feature of Holt House's design was the use of Wyatt windows in the south elevation.