Born in Lawrence, Massachusetts, Caffrey received an Artium Baccalaureus degree from the College of the Holy Cross in 1941.

[1] He was a lieutenant in the United States Army Signal Corps, Intelligence Branch.

He was Chief of the Civil Division of the Office of United States Attorney for the District of Massachusetts from 1955 to 1959, and First Assistant United States Attorney in that office from 1959 to 1960.

[2] Caffrey received a recess appointment from President Dwight D. Eisenhower on October 13, 1960, to a seat on the United States District Court for the District of Massachusetts vacated by Judge William T. McCarthy.

His service terminated on October 6, 1993, due to his death in West Palm Beach, Florida.