Until the life of Elizebeth Smith Friedman came to light, the history of American intelligence was missing a key figure. Now we know how she and her husband, William Friedman, became the progenitors of American code-making and codebreaking. 

They started out more than a century ago, as the United States prepared to enter World War I. Together, they gave birth to an empire of intelligence. By the start of World War II, their genius had engendered an army of human computers – mostly women working with pencil and paper – deciphering the secrets of the Axis and the Soviets alike. And at the dawn of the Cold War, their pioneering efforts laid a cornerstone for the foundation of the National Security Agency.

They likely never imagined what their labors would lead to – though they had their fears. Following his retirement in 1955, William Friedman warned that the NSA's “secrecy virus” would one day infect the American body politic.

The NSA, chartered in November 1952 with a secret order signed by President Harry S Truman, is an invaluable part of American intelligence. Its electronic eavesdropping is essential to the defense of the United States. But its powers exemplify the constant tension between security and liberty in our republic. It has proved to be a recurring threat to American civil rights by spying on American citizens. And its own breaches of security have left Americans vulnerable to espionage

In the Cold War, the NSA became America’s biggest intelligence service. Its mission was to intercept and decode the secret communications of foreign nations — in particular, Russia, China and North Korea — and stop other nations from doing the same to the United States. It got help: the FBI burglarized foreign embassies in Washington, and the CIA suborned clerks and communicators abroad to steal the ciphers of enemies and allies alike. But the NSA struggled mightily to crack the seemingly unbreakable codes of communist foes in Moscow, Beijing and Pyongyang.

After the Cold War, the advent of the internet, emails and cell phones increased the complexity of the NSA’s tasks exponentially. It took on a multitude of shadowy threats, including international terrorism and the proliferation of weapons of mass destruction. In the 21st century, its mission expanded to the realms of cyber warfare, like using malware to attack targets like Iran’s nuclear-weapons programs and trying to defend America’s computer systems from attack. 

The NSA has shown over the years that it is powerful when playing offense, but fallible at playing defense. Today, Russian spies have penetrated the American government to its core, and the NSA bears a large share of the blame for that still-unfolding disaster.

The creation of the NSA seven decades ago followed a series of cascading failures – and one resounding triumph. During World War II and in the war’s immediate aftermath, American intelligence was fragmented and fragile. It had failed to foresee the Japanese attack at Pearl Harbor. The Kremlin’s spies had recruited American agents at the Manhattan Project. They had moles in high places, including at the State Department, the Treasury Department and the wartime civilian intelligence agency, the Office of Strategic Services. 

American codebreakers at Arlington Hall, home of the codebreaking Army Security Agency, the NSA’s predecessor, were acolytes of the Friedmans and their painstaking methodologies. They slowly began to understand the scope of Soviet espionage. They decrypted and decoded Soviet intelligence cables sent to its spies in the United States during the war. In 1946, they broke a message that included the names of scientists working on the atomic bomb, proof of the Soviet penetration of the most secret wartime project in existence. 

But soon enough, a mole at Arlington Hall, a linguist named William Weisband, warned the Kremlin that the Americans were reading their messages. The Kremlin changed its codes – but not before the American codebreakers had dealt a blow to Soviet espionage in America by unmasking many of its spies.

William Friedman, who held the highest civilian rank at the early NSA, wrote the training manuals for the new generation of codebreakers and secured his reputation as the father of American cryptology. Before he retired in 1955, he laid the groundwork for one of the boldest operations in the history of American intelligence.

One of Friedman’s closest friends in cryptologic circles was Boris Hagelin, the Swedish founder of Crypto AG, a Swiss company that dominated the business of manufacturing encryption machines. Friedman proposed that Hagelin would sell his state-of-the-art devices only to a handful of American allies. Other countries would get less sophisticated machines – and the Americans would have the keys to unlock them. Hagelin would be paid a fortune in return. When Crypto made a new electronic encryption machine in the 1960’s, its circuits and algorithms were designed by the NSA. 

Then the CIA and the West German intelligence service, the BND, secretly bought Crypto through a shell company. It was a spectacular hack. 

Russia, China, and North Korea didn’t buy Crypto machines. But the company’s client list covered most of the rest of the world. Its biggest customers included Saudi Arabia, Iran, Egypt, Iraq, Libya, Jordan, Indonesia and South Korea. By the early 1970’s, the secret deal Friedman had conceived had evolved to the point where the NSA had easy access to much of the world’s encrypted messaging.

The NSA’s intelligence-gathering dragnet was immense. From its first days, the agency screened almost every international telegram sent from the United States, courtesy of the cable companies transmitting them. This program, codenamed SHAMROCK, grew exponentially to include the capacity to intercept telephone calls and telex messages, which the NSA swept up from satellite and microwave relay stations. 

The NSA had only a top-secret Presidential directive, not an act of Congress, as its foundation. This was a shaky basis. American cryptologists worked without the knowledge of the American public – and without knowing if what they were doing was entirely lawful. William Friedman, after his retirement, had voiced fears that the NSA was collecting too much information and doing it in ways that could pose a threat to democracy. The NSA, after all, was created to gather foreign intelligence, not to spy on United States citizens.

His fears were realized when the NSA spied on Americans in the 1960’s and 1970’s. It did so on orders from Presidents Lyndon B. Johnson and Richard M. Nixon. These programs were a foreshadowing of the NSA’s domestic spying campaigns authorized by President George W. Bush after the 9/11 attacks. 

The NSA first created a political target list of Americans in the long hot summer of 1967. It was driven by Johnson’s groundless suspicions, shared by Nixon, that Moscow and Beijing were secretly supporting Americans leading nationwide  protests against the Vietnam War and racist policing in the United States. Those two presidents saw opposition to their most politically charged policies as part of an international communist conspiracy. 

The NSA placed at least 1,650 Americans under electronic surveillance – including some of the most prominent figures in civil rights and journalism. Their names were, for the most part, supplied by the FBI and the CIA. An official NSA history of those years notes that the agency’s directors knew the surveillance operation was “disreputable if not outright illegal.” It was, in fact, in direct violation of the Fourth Amendment’s prohibition against warrantless searches.

The operation was revealed, in part, by a Senate investigation of CIA, FBI and NSA abuses conducted in the wake of Watergate. The investigation was led by Senator Frank Church, who concluded in 1976 that “[The NSA’s] capability to monitor anything…could be turned around on the American people. And no American would have any privacy left. There would be no place to hide. If a dictator ever took charge in this country, the technological capability that the intelligence community has given the Government could enable it to impose total tyranny.” Thirty-seven years later, in 2013, the NSA disclosed that the targets of its illegal surveillance had included Dr. Martin Luther King Jr., Muhammad Ali, the New York Times journalist Tom Wicker – and Senator Frank Church.

After the Church hearings, the NSA's mission was redefined in a 1981 presidential order: to collect “foreign intelligence or counterintelligence” while not “acquiring information concerning the domestic activities of United States persons.” Congress created the Foreign Intelligence Surveillance Court to require warrants for NSA wiretaps on American soil. But barely 20 years later warrantless NSA surveillance would again violate Americans’ constitutional rights.

As the Cold War neared an end, the NSA fought battles on two fronts: the continuing assault by Soviet espionage and the global power of the Internet. 

A host of American spies working against the NSA for Moscow were arrested in the 1980’s. Three cases in particular undermined American national security. Senior Navy warrant officer John Walker and a ring of friends and family stole NSA encryption keys and other cryptographic secrets that helped give Moscow a clear view of American military operations during and after the Vietnam War. A former NSA analyst, Ronald Pelton, betrayed a sprawling effort to track Soviet naval and military communications worldwide. And an FBI turncoat, Robert Hanssen, revealed a huge NSA/FBI operation to bug the Soviet embassy in Washington, destroying ten years and hundreds of millions of dollars of work.

By the time the Internet was a public phenomenon in 1995, the NSA was falling behind the technological curve. It could no longer easily tap fiber-optic cables or unlock a new wave of commercially available email encryption systems. “By the late 1990s,” recalled General Michael V. Hayden, the NSA’s director from 1999 to 2005, “the explosion of modern communications in terms of volume, variety, velocity threatened to overwhelm us.” In January 2000, the agency’s computers crashed. The NSA was unable to issue reports on intercepted foreign telephone, cable and radio messages for three days before it rebooted.

After the attacks of September 11, 2001, the NSA, on orders from President Bush, resumed warrantless wiretapping in the United States. It bypassed the Foreign Intelligence Surveillance Court in the name of secrecy. In the name of counterterrorism, under the codename STELLAR WIND, it created a database of American citizens’ email, telephone conversations, financial transactions and internet activity. 

In March 2004, the newly appointed deputy Attorney General, James M. Comey, learned of the program and questioned its constitutionality. So did the FBI director, Robert S. Mueller III. Mueller confronted the President and said he would resign unless the operation was changed to conform to the Constitution. Bush modified the program, but when The New York Times revealed aspects of STELLAR WIND in December 2005, it created a whirlwind of controversy over the powers of the NSA.

That storm erupted again in 2013 after Edward Snowden, an NSA contractor, began to disclose some of the agency’s most secret surveillance programs. Under the legal provisions of the PATRIOT Act, the NSA was obtaining a copy of everything carried along major domestic fiber optic cable networks. It was wiretapping America.

Snowden revealed programs including PRISM, which collects internet communications from companies including Google, Facebook and Microsoft, and DISHFIRE, which collects 200 million text messages a day. He said he was a whistleblower motivated by a desire to expose abuses of power, including the fact that American telephone companies gave the NSA millions of Americans’ phone records daily. But he fled to Russia and faced an espionage indictment on the heels of his revelations. The NSA has never fully explained how a lowly systems operator like Snowden could steal terabytes of top-secret data from its vaults.

Another blow fell on the NSA in 2016, when a previously unknown group calling itself the Shadow Brokers stole and revealed online the most powerful tools used by the NSA’s hacking division, Tailored Access Operations, which had created the digital weapons that disabled Iran’s uranium-enrichment centrifuges and sought to disrupt North Korea’s missile programs.

The NSA’s 21st-century cyber operations against foreign enemies were increasingly aggressive. But they could not protect the United States against repeated attacks of cyber espionage. Over the past five years, to list the most egregious cases, China stole security-clearance files on 22 million Americans in 2015, including uncounted members of the intelligence community; Russia monkey-wrenched the 2016 presidential election in support of Donald Trump. In December 2020, the Russians penetrated the American government at nearly every level with a bold cyber espionage attack, striking the Pentagon, the Homeland Security Department, the State Department and the national nuclear laboratories, among many other targets. 

The consequences of the breach are still unfolding at this writing, but they appear almost incalculable; at first glance, it appears to be one of the greatest intelligence failures of the century. Despite the NSA’s spending many billions of dollars on cyber defense, it was unaware of the attack until it was notified by private-sector experts.

The 21st-century NSA has gone far beyond its original mission of breaking enemy codes. It is the pointed end of the spear of military and intelligence operations in the information age. But it also must serve as a shield against the increasingly sophisticated cyber operations of Russia and China. If it cannot fulfill that role, it will have to be reinvented for a new era of political warfare.

Tim Weiner is the Pulitzer Prize-winning author of The Folly and The Glory: America, Russia, and Political Warfare, 1945-2020.