Dexter was settled beginning in 1801 by Ebenezer Small, David Smith, and others from New Hampshire, and was originally called Elkinstown.

When incorporated as a town in 1816, it named itself after Judge Samuel Dexter,[3] who was then running for governor of Massachusetts (of which Maine was still a part).

[4] The town grew because of its location on the East Branch of the Sebasticook River, which provided excellent water power for mills.

The Dexter Historical Society today uses the building which replaced it in 1854 as part of its museum complex.

The stream would also power five woolen mills, the oldest and largest of which was established by Amos and Jeremiah Abbott in 1836.

Amos Abbott & Company, which closed in 1975, was the only textile mill in the United States owned by one family for such a long period.

[5] In the 1960s, the town's name became familiar throughout New England because of the pervasive log cabin style factory outlets of the Dexter Shoe Company, founded in a vacant Dexter woolen mill in 1958 by Harold Alfond.

It is also Dexter's oldest house of worship, built in 1829, but given a new steeple and vestibule by Boston architect Thomas W. Silloway in 1869.

Five buildings in Dexter are listed on the National Register of Historic Places, including the Dexter Grist Mill; Universalist Church; Abbott Memorial Library by Boston architect J. Williams Beal; the Bank Block by Bangor architect George W. Orff; and "Zion's Hill", the Ralph Owen Brewster house by Portland architectural firm J. C. & J. H.

[7] In 1848, the town was struck by a tornado which tore large trees out by their roots and destroyed even the strongest buildings.