His father earned a Ph.D. in mathematics at Johns Hopkins University in 1923 and served for three decades as a professor at the United States Naval Academy in Annapolis, Maryland.
The elder Scarborough published Numerical Mathematical Analysis (1930), a book considered fundamental to the development of computers in the 1940s.
[7] Scarborough earned his B.A at the University of North Carolina at Chapel Hill in 1954, and soon after, he married Patricia Estelle Carruthers.
at Cornell and completed his doctorate under Fletcher Melvin Green at the University of North Carolina, Chapel Hill in 1962.
The records of his work, 27 feet and 8500 documents, including materials associated with the Citizens' Councils, are archived at the University of North Carolina libraries.