[1] Wilmerding was born in Boston, Massachusetts, on April 28, 1938, and was descended from prominent families in old New York City social circles.

[4] After his mother's death, his father remarried, to Katharine (née Salvage) Polk (1914–2003),[6] the daughter of Samuel Agar Salvage and widow of Frank Lyon Polk Jr.[7][8] His maternal grandparents were James Watson Webb (1884–1960)[9] and Electra Havemeyer Webb (1888–1960),[10][11] co-founders of the Shelburne Museum, which showcases the family's "collection of collections" of early American homes and public buildings, including a general store, meeting house, log cabin, and a steamship.

[14] Then, from 1977 to 1983 he served as senior curator at the National Gallery of Art, in Washington, D.C., and as its deputy director under J. Carter Brown from 1983 to 1988.

[14] Wilmerding began collecting art while still a student at Harvard, purchasing the 1857 painting Stage Rocks and Western Shore of Gloucester Outer Harbor by Fitz Hugh Lane during his senior year for $3,500.

[20] At the May 2004 opening of the National Gallery of Art's exhibit American Masters From Bingham to Eakins: The John Wilmerding Collection, Wilmerding announced that his entire collection would remain at the Gallery in perpetuity,[2] including works by such artists as Martin Johnson Heade, Fitz Henry Lane, John F. Peto, Joseph Decker, Winslow Homer, Thomas Eakins, Frederic Edwin Church, George Caleb Bingham, and John F. Kensett, and featuring certain of his favorite works by artists who visited and painted Maine's Mount Desert Island in Acadia National Park, where he summered for many years.