Located mainly on Lyme Street south of Interstate 95, the village, settled in the mid-17th century, has an architectural history dating to the early 18th century, flourishing as a shipbuilding center and home to many ship captains.

The village center has since the early days been arrayed along what is now called Lyme Street, which runs parallel to the Lieutenant River shortly before its mouth at the Connecticut River.

The artistic legacy is continued by the presence of the Lyme Art Association and the Lyme Academy College of Fine Arts in the village.

It also includes a few buildings north of I-95, including the Old Lyme Inn and the Florence Griswold House and Museum, the historic center of the art colony.

Houses in the district range across the architectural spectrum from the early 18th century to the late 19th century, and including the Peck Tavern on Sill Lane, an 18th-century tavern that is now a private residence.