He is best remembered for collecting, editing, and publishing the Paston Letters, describing the life and political scheming of the gentry in Medieval England.
Fenn was born 26 November 1739, the son of a surgeon, and was educated at the grammar schools of Scarning and Bottisdale (Botesdale) from which he was admitted to Caius College, Cambridge.
Between 1768 and 1775 Fenn helped William Whittingham to publish the remaining parts of the continuation of Francis Blomefield's History of Norfolk.
The first two volumes of Fenn's edition of the famous letters appeared in January 1787, dedicated to King George III, and caused a literary sensation.
He was buried with his wife at the Frere family's church of St Bartholomew, Finningham, Suffolk, where there was erected a significant marble monument to them both by John Bacon.