The Rumford House is a 2+1⁄2-story wood-frame structure, five bays wide, with a gambrel roof and a large central chimney.

His grandson, Benjamin Thompson Jr., later to become Count Rumford, was born on March 26, 1753, in the west end of this house, which his parents had lived in since their marriage.

He eventually settled for a time in the Holy Roman Empire, where he acquired the title of Count Rumford, and where he performed his groundbreaking research into the design of heating systems.

While in Munich, Thompson oversaw reforms of the indigent welfare system of Bavaria, and designed the city's famous English Garden.

Although he never returned to the country of his birth, he did endow a professorial chair at Harvard College, and established the Rumford Prize, an annual award given by the American Academy of Arts and Sciences.