Although unknown to the public, and even to Presidents Franklin D. Roosevelt and Harry S. Truman, these programs were of importance concerning crucial events of the early Cold War.
These included the Julius and Ethel Rosenberg spying case (which was based on events during World War II) and the defections of Donald Maclean and Guy Burgess to the Soviet Union.
[12][13][14] The VENONA[16][17] Project was initiated on February 1, 1943, by Gene Grabeel,[18] an American mathematician and cryptanalyst, under orders from Colonel Carter W. Clarke, Chief of Special Branch of the Military Intelligence Service at that time.
[22] This message traffic, which was encrypted with a one-time pad system, was stored and analyzed in relative secrecy by hundreds of cryptanalysts over a 40-year period starting in the early 1940s.
The Soviet company that manufactured the one-time pads produced around 35,000 pages of duplicate key numbers, as a result of pressures brought about by the German advance on Moscow during World War II.
The Soviet systems in general used a code to convert words and letters into numbers, to which additive keys (from one-time pads) were added, encrypting the content.
[25] However, cryptanalysis by American code-breakers revealed that some of the one-time pad material had incorrectly been reused by the Soviets (specifically, entire pages, although not complete books), which allowed decryption (sometimes only partial) of a small part of the traffic.
Generating the one-time pads was a slow and labor-intensive process, and the outbreak of war with Germany in June 1941 caused a sudden increase in the need for coded messages.
Claims have been made that information from the physical recovery of code books (a partially burned one was obtained by the Finns) to bugging embassy rooms in which text was entered into encrypting devices (analyzing the keystrokes by listening to them being punched in) contributed to recovering much of the plaintext.
[33] Identities soon emerged of American, Canadian, Australian, and British spies in service to the Soviet government, including Klaus Fuchs, Alan Nunn May, and Donald Maclean.
Among those identified are Julius and Ethel Rosenberg, Alger Hiss, Harry Dexter White (the second-highest official in the Treasury Department), Lauchlin Currie[35] (a personal aide to Franklin Roosevelt), and Maurice Halperin[36] (a section head in the Office of Strategic Services).
According to authors John Earl Haynes and Harvey Klehr, the Venona transcripts identify approximately 349 Americans who they claim had a covert relationship with Soviet intelligence, though fewer than half of these have been matched to real-name identities.
While Navy investigators thought it was a routine systems upgrade others were not so sure and later it was attributed to two Soviet spies: American William (Bill) Weisband, a linguist at Arlington Hall and British Kim Philby in the SIS.
[40] At this time some information was trickling in from US intercepts and rare overflights near the East-West border; but with a dearth in intelligence, not even a hint was received of the North Korean attack (approved by Stalin) on South Korea in June 1950.
[43] Military historian Eduard Mark[44] and American authors Herbert Romerstein and Eric Breindel concluded it was Roosevelt's aide Harry Hopkins.
The Venona evidence indicates unidentified sources code-named "Quantum" and "Pers" who facilitated transfer of nuclear weapons technology to the Soviet Union from positions within the Manhattan Project.
According to Alexander Vassiliev's notes from KGB archive, "Quantum" was Boris Podolsky and "Pers" was Russell W. McNutt, an engineer from the uranium processing plant in Oak Ridge.
[58] The Venona decryptions also identified Soviet spy Harry Gold as an agent of the KGB who stole blueprints, industrial formulas, and methods on their behalf from 1935 until ultimately confessing to these actions in 1950.
He became a Soviet informant after beginning his studies at the University of Cambridge, where he and his classmates (Kim Philby, Anthony Blunt, and Donald Maclean) began developing ideals against a capitalist society.
[citation needed] In 1950, George Ronald Richards was appointed ASIO's deputy-director of operations for Venona, based in Sydney, charged with investigating intelligence that uncovered the eleven Australians identified in the cables that had been decoded.
Army Chief of Staff Omar Bradley, concerned about the White House's history of leaking sensitive information, decided to deny President Truman direct knowledge of the project.
Some of the earliest detailed public knowledge that Soviet code messages from World War II had been broken came with the release of Chapman Pincher's book, Too Secret Too Long, in 1984.
However, the first detailed account of the Venona project, identifying it by name and making clear its long-term implications in post-war espionage, was contained in MI5 assistant director Peter Wright's 1987 memoir, Spycatcher.
As the Moynihan Commission wrote in its final report: A balanced history of this period is now beginning to appear; the Venona messages will surely supply a great cache of facts to bring the matter to some closure.
[citation needed] Controversy arose in 2009 over the Texas State Board of Education's revision of their high school history class curricula to suggest Venona shows Senator Joseph McCarthy to have been justified in his zeal in exposing those whom he believed to be Soviet spies or communist sympathizers.
[80] Critics such as Emory University history professor Harvey Klehr assert most people and organizations identified by McCarthy, such as those brought forward in the Army-McCarthy hearings or rival politicians in the Democratic party, were not mentioned in the Venona content and that his accusations remain largely unsupported by evidence.
Intelligence historian Nigel West believes that "Venona remain[s] an irrefutable resource, far more reliable than the mercurial recollections of KGB defectors and the dubious conclusions drawn by paranoid analysts mesmerized by Machiavellian plots.
Writers Walter and Miriam Schneir, in a lengthy 1999 review of one of the first book-length studies of the messages, object to what they see as the book's overconfidence in the translations' accuracy, noting that the undecrypted gaps in the texts can make interpretation difficult, and emphasizing the problem of identifying the individuals mentioned under cryptonyms.
Victor Navasky, editor and publisher of The Nation, has also written several editorials highly critical of Haynes' and Klehr's interpretation of recent work on the subject of Soviet espionage.
"[13] According to historian Ellen Schrecker, "Because they offer insights into the world of the secret police on both sides of the Iron Curtain, it is tempting to treat the FBI and Venona materials less critically than documents from more accessible sources.