He was in the United States Army from 1967 to 1973, achieving the rank of Reserve Captain.
He was a legislative assistant to United States Senator Strom Thurmond from 1970 to 1972.
He was an Assistant United States Attorney of the Criminal Division in Greenville, South Carolina from 1972 to 1976.
He was an Assistant United States Attorney of the Civil Division in Columbia, South Carolina from 1983 to 1986.
Herlong was confirmed by the United States Senate on May 9, 1991, and received his commission on May 14, 1991.