Built in 1804, it is one of the area's most prominent examples of Federal period architecture, albeit with substantial early 20th-century Colonial Revival alterations.

The Abbot House stands on the north side of Nashville Street, facing south.

The main entrance is in the center, sheltered by a rectangular portico supported by four large fluted columns and topped by a balustrade.

It was owned 1854–1892 by George Perham, under whose ownership the exterior was given an extensive Victorian treatment, which included a three-story tower in the central bay.

Purchased in 1905 by William Spaulding, he removed most of the Victorian alterations, while adding Colonial Revival features, including the present portico.