The home was built about 1721 by Nathaniel Ladd as one of the state's first brick houses, and was subsequently clapboarded three decades later.

Also born in the house was Founding Father Nicholas Gilman, Jr., a signer of the United States Constitution and U.S. senator from New Hampshire.

The house is a rambling frame structure, consisting of a main block and a series of additions.

A further addition, originally designed as a caretaker's residence, was added after the Society of the Cincinnati acquired the house in 1902.

The interior portions of the main house feature original woodwork, including paneling, deep window seats, and fluted pilasters.