His father, after working in the store in his boyhood, went to Boston for several years, where, according to Judd's sister's biography, he became a voracious reader, returning to the family business, but then becoming editor of the Hampshire Gazette.

Sylvester Judd III studied at Hopkins Academy in Hadley, Massachusetts, where he was president of the Literary Society and delivered the valedictorian address.

While a student, on April 4, 1838, Judd traveled to Concord, Massachusetts to meet Ralph Waldo Emerson after reading his essay "Epic Poetry".

[3] Judd may have been in the audience on August 31, 1837, and heard Emerson's commencement speech to the Phi Beta Kappa Society known as "The American Scholar".

[4] The couple married on August 31, 1841; they had three daughters: Jane Elizabeth (September 26, 1844), Frances Hall (June 28, 1847), and Apphia Williams (March 16, 1853).

His readings included the poetry of Jones Very, the writings of Thomas Carlyle, Johann Gottlieb Fichte, Robert Owen, Goethe's Wilhelm Meister's Apprenticeship, Goethe's Conversations with a Child by Bettina von Arnim, and the writings of the foremost Unitarian ministers of their day, William Ellery Channing and Ralph Waldo Emerson.

Powers finds the child characterization of Margaret in the novel the prototype for Nathaniel Hawthorne's character of Pearl, Hester Prynne's daughter, in the 1850 novel The Scarlet Letter.