After attending local schools, he graduated from Norwich Military Academy in 1824, and entered the University of Vermont in Burlington.

He moved to Derby Line, Vermont, in 1828, where he engaged in agricultural and mercantile pursuits, which took him down the Connecticut River valley and into Canada.

He was one of the original incorporators of the Connecticut and Passumpsic Rivers Railroad, which was planned to run almost the entire length of the state on the eastern border.

Baxter also frequently visited the regiments in the area immediately surrounding Washington, D.C., watching out for a son who had joined the 11th Vermont Infantry, and sponsoring others in their efforts to get promoted.

During the bloody Battle of the Wilderness in May 1864, Baxter and his wife spent so much time in the hospitals in and around Fredericksburg, Virginia, tending to wounded soldiers, that they themselves suffered from exhaustion and eventually had to leave to recuperate.

The most notable was Jedediah Hyde Baxter, who served as Surgeon General of the United States Army.