The Grafton Village Historic District occupies approximately the same area.

The CDP is in northern Windham County, southeast of the center of Grafton.

It sits in the valley of the Saxtons River, where it is joined by its South Branch.

Vermont Route 121 serves as Grafton's Main Street.

It leads southeast 11 miles (18 km) to Bellows Falls on the Connecticut River and northwest 14 miles (23 km) to Londonderry.