It was incorporated in 1773 by Governor John Wentworth and named for George Jaffrey, member of a wealthy Portsmouth family.

Village prosperity would be expressed in fine early architecture, including the Town Meetinghouse, built in 1775.

Beginning in the 1840s, the area's scenic beauty attracted tourists, and several summer hotels were built at the base of Mount Monadnock, enduringly popular with hikers.

Some who scaled the summit were Ralph Waldo Emerson, Henry David Thoreau and Rudyard Kipling.

Jaffrey was the setting for a 1950 biography by Elizabeth Yates entitled Amos Fortune, Free Man, winner of the 1951 Newbery Medal.

Amos Fortune was an African-born slave who purchased his freedom and that of his wife, and established a tannery in the village.

He is buried in the local cemetery, as are bandbox craftswoman Hannah Davis and author and summer resident Willa Cather.

Mount Monadnock, elevation 3,165 feet (965 m) and the highest point in Jaffrey as well as Cheshire County, is in the northwest.