The town center is located on terraces on the west bank of the Pemigewasset River, with its commercial core extending along Main Street (United States Route 3).

Its most prominent feature is a fountain, depicting a Boy Scout kneeling with cupped hands to hold water; it was designed by George Borst, a summertime resident of Plymouth, and placed in 1933.

It was here that the town's first colonial meeting house was built, on whose site the 20th-century Congregational Church now stands.

Just to its north stands Plymouth Town Hall, built in 1890 to a design by New Hampshire architect C. Willis Damon to also serve as a county courthouse.

Adjacent to the town hall is the Old Grafton County Courthouse, one of the state's oldest civic buildings, built in 1774.