He settled with his family in the Grove community in southeastern James City County, about 7 miles (11 km) east of Williamsburg.
During the administration of President Franklin D. Roosevelt, he served as a New Deal public health physician, and built the two-bed Blayton Maternity Hospital for the African-American community.
He also opened a sandwich and soda shop to serve young people shut out of other facilities by segregation.
In 1999, he was honored by the Williamsburg Community Health Foundation as one of the first three recipients of its Healthcare Heroes Recognition Award.
He was co-chairman of the campaign to raise $250,000 for the construction of Quarterpath Park, director of health and safety for the Peninsula Boy Scouts for 25 years, and a life member of the NAACP.
Dr. Blaytons's daughter Betty Blayton-Taylor was a well-known and influential African American artist and educator in New York City.