After graduating from high school, Sanders initially began his military career by joining the United States Air Force in February 1954.

Sanders enlisted in the United States Navy on 9 May 1958, and after receiving his clothing issue at Charleston, South Carolina, he was transferred to Naval Air Technical Training Center (NATTC) at NAS Memphis, Tennessee, to attend Aviation Electronics Technician "A" School.

Upon graduation from "A" school at NATTC Glynco, Georgia, he reported for duty as an enlisted aircrewman on EC-121 type aircraft.

While serving as detachment chief petty officer at Da Nang, Vietnam, Sanders received two Air Medals and the Combat Action Ribbon.

Sanders was transferred to NAS Pensacola, Florida, in October 1977 where he served as Air Operations Maintenance Chief until March 1978.

Sanders transferred to the Naval Education and Training Program Development Center Saufley Field at Pensacola, Florida in February 1982 to serve as the Special Projects Division Officer, then on 16 July 1982, the new Chief of Naval Operations, Admiral James D. Watkins, announced that he had selected Sanders as the fifth Master Chief Petty Officer of the Navy.

During Sanders' tenure as the MCPON, the grooming standards for sailors made a major change, as males were no longer authorized to wear beards.