A Wake Forest graduate, Scarborough was used sparingly by the Washington Senators before World War II.

Scarborough often received attention from the press in New York and Boston due to his off-season work as a pickle salesman for the Mt.

[2] Following his playing career, Scarborough moved to Mount Olive, North Carolina, where he opened an oil and supply company.

Later he scouted for the Baltimore Orioles, California Angels and Milwaukee Brewers organizations, and also helped establish a baseball program at Mount Olive College.

Scarborough's grandson Garrett Blackwelder played basketball at East Carolina University from 1996 to 2000 and holds several shooting records.