Vermont Route 108 passes through the center of town, leading north to Enosburg Falls and south to Jeffersonville.

Vermont Route 36 leads west from VT 108 in the village center to St. Albans, the Franklin County seat.

[4] The town is part of the Missisquoi River watershed, draining to Lake Champlain.

The Cold Hollow Mountains occupy the eastern end of the town, with a high point of 3,261 feet (994 m) just north of the Lamoille County line.

The town history began when Joseph Baker of Westborough, Massachusetts, the namesake for the village, bought 10,000 acres in 1791.

Initial settlers were his son-in-law, Stephen Maynard, and his nephew Jonas Brigham, along with their families.

Five acres were deeded for the development of a town common and burying ground in 1804 by Joseph Baker.

Maynard built a Federal style house north of the village to serve as the first post office in 1811.

The Bakersfield Academic Association was established in 1839, which built a three-story building to house a Methodist church and the South Academy.

[6] Jacob Spaulding was hired as headmaster and Mary, his wife, was a drawing instructor and preceptress.

[5] The academies closed after a loss of students due to westward expansion, the creation of central public high schools, and the American Civil War.