1

COLLECTIVE BLINDNESS

I

On August 9, 2001, Zacarias Moussaoui, a thirty-three-year-old French Moroccan, enrolled at the Pan Am International Flight Academy in Eagan, Minnesota. This facility, complete with a high-fidelity simulator, provided a comprehensive training program on how to fly commercial airliners. On the surface, Moussaoui seemed like any of the other men who wanted to learn how to fly jumbo jets. He was friendly, inquisitive, and seemingly wealthy. And yet over the course of two days, his instructors became suspicious. He paid for the bulk of the $8,300 course with $100 bills. He seemed unusually interested in the cockpit doors. He kept asking about flight patterns in and around New York.

The staff became so doubtful that two days after Moussaoui enrolled at the school, they reported him to the FBI in Minnesota. He was duly arrested. The FBI questioned him and applied for a warrant to search his apartment, but couldn’t show probable cause. Crucially, they failed to connect what they knew about Moussaoui with the broader threat of Islamic extremism. Here was a man with a suspected immigration violation enrolling at a flying school, asking unusual questions, and paying in cash. Weeks later would be the biggest terrorist attack in history.

IN THE MONTHS AFTER 9/11, multiple investigations were launched to work out why such an audacious plot was not foiled by America’s intelligence agencies, a group totaling tens of thousands of personnel and in command of a combined budget of tens of billions of dollars. Many of these investigations concluded that the inability to prevent the attack represented a catastrophic failure.

The CIA came in for much of the severest criticism. This is the body, after all, that had been specifically created to coordinate the intelligence community’s activities against threats, especially those originating from abroad. From the time the attacks were approved by Osama bin Laden in late 1998 or early 1999, the agencies had twenty-nine months to thwart the plot. They didn’t. Richard K. Betts, director of the Saltzman Institute of War and Peace Studies, called it a second Pearl Harbor for the United States. Milo Jones and Philippe Silberzahn, two leading intelligence experts, described it as the greatest debacle in the history of the CIA.

One might be tempted to concur, given the clues that had accumulated in the years before 9/11. Al Qaeda had broken its religious taboo on suicide bombing as early as 1993. Bin Laden, a Saudi-born son of a wealthy businessman turned religious zealot, constantly cropped up in raw intelligence reports about Arab terrorist groups. Richard Clarke, a former national coordinator for security under Ronald Reagan, said, There seemed to be some organizing force and maybe it was he. He was the one thing that we knew the terrorist groups had in common.

Bin Laden publicly declared war on the United States on September 2, 1996, saying in a recorded message that he wanted to destroy the oppressor of Islam. His strident message was gaining ground among disenfranchised Muslims. Half of terrorist organizations last less than a year, and only 5 percent survive a decade. Al Qaeda had longevity. It was an outlier.

The idea of an airplane being used as a weapon had been circulating for almost a decade. In 1994, an Algerian group hijacked a plane in Algiers and reportedly intended to blow it up over the Eiffel Tower. Later that year, Tom Clancy penned a thriller about a Boeing 747 being flown into the U.S. Capitol Building. It debuted at number one on the New York Times bestseller list. In 1995, police in Manila filed a detailed report about a suicide plot to crash a plane into CIA headquarters.

In 1997, Ayman Al Zawahiri—bin Laden’s deputy—underscored the intent of Al Qaeda by inciting a massacre of tourists in Egypt, an atrocity that left sixty-two dead, including children. One Swiss woman witnessed her father’s head being severed from his body. The Swiss federal police concluded that bin Laden had financed the operation. Unlike previous terrorist groups, Al Qaeda seemed committed to maximizing human suffering, including that of innocents.

In 1998, bin Laden went even further in his thirst for violence against the United States. In a widely published fatwa, he said, To kill the Americans and their allies—civilians and military—is an individual duty for every Muslim who can do it in any country in which it is possible to do it. On August 7, simultaneous Al Qaeda bombings in Nairobi and Dar es Salaam killed 224 people and wounded over 4,000. The first was achieved with an explosive device containing more than 2,000 pounds of TNT.

On March 7, 2001, six months before the attack on the World Trade Center, the Russians submitted a report on Al Qaeda providing information on thirty-one senior Pakistani military officers actively supporting bin Laden and describing the location of fifty-five bases in Afghanistan. Soon after, the Egyptian president Hosni Mubarak warned Washington that terrorists were planning to attack President Bush in Rome using an airplane stuffed with explosives. The Taliban foreign minister confided to the American consul general in Peshawar that Al Qaeda was planning a devastating strike on the United States. He feared that American retaliation would destroy his country.

In June 2001, just a few weeks before Moussaoui enrolled at the aviation school in Minnesota, Kenneth Williams, an FBI analyst in Arizona, sent an email to colleagues. It said, The purpose of this communication is to advise the bureau and NY [New York] of the possibility of a coordinated effort by Osama bin Laden to send students to attend civil aviation universities and colleges. He advised headquarters of the need to make a record of all the flight schools in the country, interview the operators, and compile a list of all Arab students who had sought visas for training. This was to become known as the legendary Phoenix memo. Yet it wasn’t acted upon.

With so many pieces of evidence, critics were scathing that the intelligence agencies didn’t identify—let alone infiltrate—the plot. The joint senate committee concluded, The most fundamental problem … is our intelligence community’s inability to ‘connect the dots’ available to it before September 11, 2001, about terrorists’ interest in attacking symbolic American targets.

It was a damning assessment. Perhaps understandably, the CIA responded defiantly. They defended their record, arguing that it is easy to detect terrorist plots—but only with the benefit of hindsight. They pointed to the research of the psychologists Baruch Fischhoff and Ruth Beyth who, before the historic trip of Richard Nixon to China, asked various people to estimate the probability of different outcomes. Would it lead to permanent diplomatic relations between China and the United States? Would Nixon meet with Mao Zedong at least once? Would Nixon call the trip a success?

The visit was a triumph for Nixon, but what was remarkable was how subjects remembered their estimates. Those who thought it would be a disaster recalled being highly optimistic about its success. As Fischhoff put it, Subjects reconstructed having been less surprised by the events … than they really should have been. He called it creeping determinism.

The 9/11 plot may have seemed glaringly obvious after the event, but was it really so obvious beforehand? Was this not another case of creeping determinism? Was the CIA being condemned for an attack that was, at the time, difficult to detect amid so many other threats?

A nation like the United States is the subject of countless dangers. Terrorist groups stretch around the planet. Surveillance picks up moment-by-moment digital chatter, the vast majority of which amounts to little more than trash talk and idle threats. The agencies could investigate all threats, but this would overwhelm their resources. They would be overdiagnosing the problem, hardly an improvement. As one counterterrorism chief put it, the problem was sorting red flags in a sea of Red flags.

To the CIA and their defenders, 9/11 was not a failure of intelligence but a symptom of complexity. This debate has raged ever since. On one side are those who say that the agency missed obvious warning signs. On the other are those who say that the CIA did everything they reasonably could and that plots are notoriously difficult to detect before the event.

What few people consider is the possibility that both sides were wrong.

II

In the years after it was founded in 1947, the CIA implemented rigorous hiring policies. This was an organization that demanded the best of the best. Potential CIA analysts were put through not only a thorough background investigation, polygraph examination, and financial and credit reviews, but also a battery of psychological and medical exams. And there is no doubt they hired exceptional people.

The two major exams were an SAT-style test to probe a candidate’s intelligence and a psychological profile to examine their mental state, a CIA veteran told me. The tests filtered out anyone who was not stellar on both tests. In the year I applied, they accepted one candidate for every twenty thousand applicants. When the CIA talked about hiring the best, they were bang on the money.

And yet most of these recruits also happened to look very similar: white, male, Anglo-Saxon, Protestant Americans. This is a common phenomenon in recruiting, sometimes called homophily: people tend to hire people who look and think like themselves. It is validating to be surrounded by people who share one’s perspectives, assumptions, and beliefs. As the old saying goes, birds of a feather flock together. In their meticulous study of the CIA, Jones and Silberzahn write, The first consistent attribute of the CIA’s identity and culture from 1947 to 2001 is homogeneity of its personnel in terms of race, sex, ethnicity, and class background (relative both to the rest of America and to the world as a whole). Here is the finding of an inspector general’s study on recruitment:

In 1964, the Office of National Estimates [a part of the CIA] had no Black, Jewish, or women professionals, and only a few Catholics.… In 1967, it was revealed that there were fewer than twenty African Americans among the approximately twelve thousand nonclerical CIA employees. According to a former CIA case officer and recruiter, the agency was not hiring African Americans, Latinos, or other minorities in the 1960s, a habit that continued through the 1980s.… Until 1975, the IC [the U.S. intelligence community] openly barred the employment of homosexuals.¹

In June 1979, the agency was taken to court for failing to promote female operations officers, settling out of court the following year. A few years later, the agency paid out $410,000 to settle a gender discrimination case brought by an officer with twenty-four years of experience. In 1982, the CIA paid $1 million in a class-action case accusing the agency of the same biases. And yet the CIA didn’t significantly alter its personnel policies. Nothing really changed, one analyst said.

Talking about his experience in the CIA in the 1980s, one insider wrote, The recruitment process for the clandestine service led to new officers who looked very much like the people who recruited them—white, mostly Anglo-Saxon, middle and upper class, liberal arts college graduates.… Few non-Caucasians, few women. Few ethnics, even of recent European background. In other words, not even as much diversity as there was among those who had helped create the CIA.

At a conference in 1999 entitled U.S. Intelligence and the End of the Cold War, there were thirty-five speakers and presenters, of which thirty-four were white males. The one exception was a white female who introduced a dinner speaker. Of the three hundred people who attended, fewer than five were not white.

There are no publicly available numbers on the religious orientation of CIA officials responsible for deciding the agency’s tasking priorities, but Jones and Silberzahn state, We can assume based on what we know of Langley’s homogeneity that there were few (if any) Muslims among them. This was corroborated by a former CIA staffer, who said, Muslims were virtually nonexistent.

Diversity was squeezed further after the end of the Cold War. Legacy of Ashes, by the Pulitzer Prize–winning reporter Tim Weiner, quotes Robert Gates, director of the CIA in the early 1990s, as saying that the agency became less willing to employ people who are a little different, people who are eccentric, people who don’t look good in a suit and tie, people who don’t play well in the sandbox with others. The kinds of tests that we make people pass, psychological and everything else, make it hard for somebody [with] unique capabilities to get into the agency.

A former operations officer said that through the 1990s, the CIA had a white-as-rice culture. In the months leading up to 9/11, an essay written for the International Journal of Intelligence and CounterIntelligence commented, From its inception, the intelligence community [has been] staffed by the white male Protestant elite, not just because that was the class in power, but because that elite saw itself as the guarantor and protector of American values and ethics.

The homogeneity at the CIA led to occasional headshaking from politicians who were aware of it. They worried that the CIA was not representative of the society it was created to protect. They believed that if there were more women and ethnic minorities, it would encourage a broader population to feel able to report their concerns, and to imagine working there themselves. They wanted a more inclusive workforce. But CIA insiders always held what seemed like a trump card. Any dilution in their focus on ability, they said, would threaten national security. If you are hiring a sprint relay team, you select the fastest runners. If they are all the same color and gender, so what? To use any criteria of recruitment beyond speed is to undermine performance. In the context of national security, they said, putting political correctness above safety is not an acceptable option.

This idea that there is a trade-off between excellence and diversity has a long tradition. In the United States, it formed the basis of a seminal argument by Justice Antonin Scalia for the Supreme Court. Either you can choose diversity, he contended, or you can choose to be super-duper. If a diverse workforce, student population, or whatever emerges organically through the pursuit of excellence, that is one thing. But to privilege diversity above excellence is different. Diversity above all, the argument goes, is likely to undermine the very objectives that inspired it.

In a relay team, you end up losing the race. If you are a business, it’s even worse: you jeopardize your existence. A bankrupt company cannot sustain any workforce, diverse or otherwise. And when it comes to national security, there is a risk that you will imperil the very population you are tasked to protect. And how can that be an ethical course of action? As one former CIA analyst told me, There was a strong feeling that there should be no compromise. It didn’t make sense to ‘broaden’ the workforce—whatever that means—if it meant that we might lose our cutting edge. It wasn’t pigheadedness; it was patriotism.

As late as 2016, security experts were making the same point. In a column for the National Review, Fred Fleitz, a former CIA analyst who would become chief of staff for the National Security Council under President Trump, criticized an initiative to increase diversity at the CIA. Protecting our nation from such threats requires extremely competent and capable individuals to conduct intelligence operations and write analysis in challenging security and legal environments.… The CIA’s mission is too serious to be distracted by social-engineering efforts.

Part of the reluctance to recruit ethnic minorities was fear of counterespionage, but the skepticism went far deeper. Those who called for a broader intake were shouted down for undermining excellence. The CIA should be about the brightest and the best! Defense is too important to allow diversity to trump ability! As one observer put it, Political correctness should never be elevated above national security.

What they didn’t realize was that this was a false, and perilous, dichotomy.

III

This is a book about diversity. At one level, this might seem like a curious objective. Surely, we should aim to think correctly or accurately, not differently. One should only wish to think differently from other people when they are in the wrong. This seems like common sense.

Another seemingly commonsensical statement, made by Justice Scalia, argued that recruiting people because they are different, in one way or another, is to jeopardize performance. You should hire people because they are smart, or knowledgeable, or fast. Why would you hire people who are less knowledgeable, fast, or talented, just because they are different?

In the coming pages, we will show that both these intuitions are false, at least when it comes to the challenging problems we care most about. If we are intent on tackling our most serious questions, from climate change to poverty, from curing diseases to designing new products, we need to work with people who think differently, not just accurately. And this requires us to take a step back and view performance from a fundamentally different vantage point.

Consider the way we traditionally think about success. If you look at science or, indeed, popular literature, the focus is on individuals. How can we improve the knowledge or perceptiveness of ourselves or our colleagues? Fine books such as Peak by Anders Ericsson and Robert Pool, Sources of Power by Gary Klein, and Mindset by Carol Dweck have become bestsellers. All examine, in their different ways, how we can improve individual abilities through time.

A host of other excellent books follow this approach but in a slightly different way. Even when we have developed expertise, we can still be vulnerable to biases and quirks that undermine our capacity to make wise judgments. Thinking, Fast and Slow by Daniel Kahneman, Predictably Irrational by Dan Ariely, and Misbehaving by Richard Thaler all seek to improve performance by understanding these biases and how to guard against them.

But by focusing on individuals, there has been a tendency to overlook what we might call the holistic perspective. Consider a colony of ants. A naive entomologist might seek to understand the colony by examining the ants within the colony. Individual ants, after all, deploy a vast range of behaviors, such as collecting leaves, marching, and so on. They are busy and fascinating creatures. And yet you could spend a year, indeed a lifetime, examining individuals and learn virtually nothing of the colony. Why? Because the interesting thing about ants is not the parts but the whole. Instead of zooming in on individual ants, the only way to understand the colony is to zoom out. One step removed, you can comprehend the colony as a coherent organism, capable of solving complex problems: it builds sophisticated homes, it finds sources of food. An ant colony is an emergent system. The whole is bigger than the sum of its parts.

This book will argue that the same applies to human groups. Pretty much all the most challenging work today is undertaken in groups for a simple reason: problems are too complex for any one person to tackle alone. The number of papers written by individual authors has declined year by year in almost all areas of academia. In science and engineering, 90 percent of papers are written by teams. In medical research, collaborations outnumber individual papers by three to one.

In business, we see the same trend. A team led by Brian Uzzi, a psychologist at Kellogg School of Management, examined more than two million patents issued by the United States since 1975 and found that teams are dominant in every single one of the thirty-five categories. The same trend is seen in the marketplace. Twenty-five years ago, most equity funds were managed by individuals. Now, the vast majority are run by teams. The most significant trend in human creativity is the shift from individuals to teams, and the gap between teams and individuals is increasing with time, Uzzi writes.

And this is why the holistic perspective is so imperative. We need to think of human performance from the standpoint not of the individual but of the group. From this more rounded perspective, we’ll see that diversity is the critical factor driving what we might term collective intelligence.

There are, of course, many types of diversity. Differences in gender, race, age, and religion are sometimes classified as demographic diversity (or identity diversity). We will be focusing instead on cognitive diversity. That’s to say, variance in perspective, insights, experiences, and thinking styles. There is often (but not always) an overlap between these two concepts. People from different backgrounds, with different experiences, often think about problems differently than one another. We will analyze the precise relationship later in the book.

Cognitive diversity was not so important a few hundred years ago, because the problems we faced tended to be linear, or simple, or separable, or all three. A physicist who can accurately predict the position of the moon doesn’t need another opinion to help her do her job. She is already bang on the money. Any other opinion is false. This goes back to our commonsense intuition. Thinking differently is a distraction. With complex problems, however, this logic flips. Groups that contain diverse views have a vast, often decisive, advantage.

Another point worth noting is that these are not speculative claims; rather, they emerge from rigorous, if initially puzzling, axioms. As Scott Page, an expert in complexity science at the University of Michigan, Ann Arbor, has pointed out, these axioms apply as much to computers as to humans. As we shall see, artificial intelligence today relies on ensembles of algorithms that think differently, search differently, and encode problems in diverse ways.

Over the coming pages, the contours of a new science will emerge. Our journey will take us to unusual destinations: the death zone at the summit of Mount Everest, the American neo-Nazi movement after the 2008 presidential election, and sub-Saharan Africa at the dawn of our species. We will see why the U.S. Air Force endured so many crashes in the 1940s–50s, how the Dutch reinvented soccer, and why most diets suit almost nobody. We will look at success stories, peeling back the layers of how they happened, and examining their hidden logic. We will look at seminal failures, too. Often, looking at what went wrong provides the most vivid pointers about how to get things right.

By the end of the book, we will be equipped with a fresh perspective on how success happens, one with implications not just for governments and business, but for all of us. To harness the power of cognitive diversity is to gain a key source of competitive advantage, and the surest route to reinvention and growth. We are entering the age of