AI and Humanity

Illah Reza Nourbakhsh and Jennifer Keating

The MIT Press

Cambridge, Massachusetts

London, England

© 2019 The Massachusetts Institute of Technology

All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.

Library of Congress Cataloging-in-Publication Data

Names: Nourbakhsh, Illah Reza, 1970- author. | Keating, Jennifer, author.

Title: AI and humanity / Illah Reza Nourbakhsh, Jennifer Keating.

Description: Cambridge, Massachusetts : The MIT Press, 2020. | This work began as an experimental undergraduate course, AI & Humanity, offered in the School of Computer Science and the Dietrich College of Humanities and Social Sciences at Carnegie Mellon University--Preface. | Includes bibliographical references and index.

Identifiers: LCCN 2019029886 | ISBN 9780262043847 (hardcover) | ISBN 9780262358156 (ebook)

Subjects: LCSH: Artificial intelligence--Philosophy. | Artificial intelligence--Social aspects. | Technology--Social aspects.

Classification: LCC Q334.7 .N68 2020 | DDC 303.48/34--dc23

LC record available at https://lccn.loc.gov/2019029886

d_r0

For Mom. You have taught me to believe in the connections that I see.

—JTK

To Nikou, Mitra, and the CREATE Lab at Carnegie Mellon’s Robotics Institute. You inspire me every instant.

—IRN

Contents

Preface

Acknowledgments

1 Introduction

2 Technology and Society

3 Labor and the Self

4 (In)equality and (Post)humanity

5 Surveillance, Information, Network

6 Weaponry, Agency, Dehumanization

7 Shaping Our Future

Appendix A: Sample Assignments

Appendix B: Sample Rubrics

Appendix C: Concept Mapping Primer

Appendix D: Sample Course Syllabus

Bibliography

Index

Color Plates

List of Figures

Figure 1.1 Swiss automaton writing letters longhand, circa 1770.

Figure 2.1

Figure 3.1

Figure 4.1a, b reCAPTCHA humanity verification screenshots, circa 2017.

Figure 5.1 Bentham’s panopticon, illustrated by Willey Reveley (1791).

Figure 6.1 Data, Star Trek: The Next Generation

Figure 6.2 W. Grey Walter’s robotic turtle.

Figure 6.3 Gmail interaction from May 2017.

Figure 6.4 Gmail interaction with spam email from May 2017.

Figure 7.1

Figure C.1 Sample concept map produced by a first-year student for AI & Humanity (Carnegie Mellon University, 2017). Note the use of concept nodes, labeled relations between nodes, and annotated references to the reading.

Preface

This is not a TED Talk. In this book, you will find that our questions increasingly lead to more questions rather than to definitive answers. As we work through this thought experiment on the myriad relationships between developing artificial intelligence (AI) and humanity, we require your patience (mixed with generous doses of imagination!). We will introduce and explore the historical development of various AI systems and tools, moving from the seemingly innocuous Roomba to the considerably more sinister Harpy semiautonomous weapon system. We will consider the evolution of various AI tools and consider how these tools affect our individual relationships to systems and the ways in which these tools might impact relationships between humans (as individuals and collectives). We are at the beginning of what looks like a revolutionary period in relationships between humans and machines. How then might we anticipate the questions that will come to define this epoch? Will AI systems undermine our abilities to make decisions? Will these tools assist us in achieving scientific discoveries, health interventions, or relief of inequality beyond our wildest dreams?

We are only at the beginning of understanding what the features of this new era will be. But how might we better prepare citizens the world over with knowledge of the technological tools that continue to infiltrate and influence our lives? And what lessons might we learn from history to better prepare ourselves for the seemingly inevitable ruptures and changes already unfolding in this technological revolution? How can we anticipate certain cultural shifts and changes that might already be underway? And what tools will we need to regulate these changes to harness the good, rather than subject ourselves to the potential ill effects, of a rapidly changing, technologically integrated world?

Throughout this book, you will engage with two authors: a humanist, with educational training in literature and cultural studies, and a roboticist, educated as a computer scientist. Our disciplinary perspectives frame the course of our inquiry and serve as footholds for disrupting many mainstream discussions presently unfolding in regard to developing AI: its potential ethical implications, social impact, and cultural influence. This work began as an experimental undergraduate course, AI & Humanity, offered in the School of Computer Science and the Dietrich College of Humanities and Social Sciences at Carnegie Mellon University. It is one of a fleet of experimental courses, crossing conventional academic boundaries to attend to grand societal challenges. Through the lenses of literature, film, visual art, and memoir, students explore how people negotiate power structures and the ways in which individuals are influenced by emerging technologies in a variety of cultural and historical contexts. We consider how human relationships shape and influence the tools that we build and how these tools in turn shape how we navigate these relationships in society. For example, we consider the cotton gin. How did this technology influence demands for swaths of land and an expanded labor force, as well as contribute to an American corner on the international cotton market in the late seventeenth and early eighteenth centuries? How might this example compare to the advent of thinking machines in the example offered by the IBM Watson platform or issues pertaining to features of orientation and identification in developing semiautonomous vehicles?

We suggest the downstream impact of technological innovation on humanity only grows stronger as the tools that we use become increasingly sophisticated and integrated with modern life. As we engage with students who will become the leaders of future generations—technologists and humanists who will build, regulate, and critically consider the role of these tools—we are invested in building our readers’ and our students’ skill in framing these pertinent questions, narrowing the scope of the problems and identifying tools that will prepare us for navigating the coming chapter in human history. We are invested in building a common language to consider our shared history and in using this lexicon as a tool to analyze how rapidly advancing technological systems influence our lives.

Historically, schooling systems and universities have been poorly designed for the cross-disciplinary demands that require artful navigation of social and cultural questions. In this book, we consider how technological tools are devised and evolved to attend to specific questions and problems in society. We also interrogate how carefully we anticipate the intended uses or unintended uses of these systems as designers, engineers, programmers, consumers, and regulators. What happens when a faucet sensor is not attuned to variations in skin pigment and cannot see hands that fall beyond the spectrum of its programmed skin pigment recognition algorithm? How are drones coded to identify nonstate actors in a war zone? What features distinguish a combatant from a civilian in such circumstances? How does the language we use to describe AI systems that assist humans or even outpace our abilities use literary tropes or metaphor to imply a hyperbolic level of capability?

Our vision in this book is to equip readers with a robust vocabulary and fluency of technological advancement to ensure comfort and dexterity in attending to the emerging questions of our time. We want to help build a shared intellectual space that can be combined with structured opportunities to consider the world as it changes and to envision a world in which AI systems are harnessed for the betterment of society, rather than perpetuating its ills. Today we inhabit a boundary space where we test the agility of our social and political conventions. We face a wave of revolutions last seen during the rise of industrialization in the nineteenth century, the machine age at the turn of the twentieth century, and the advent of the digital economy in the late twentieth century. Like in recent histories, our political and social systems are not equipped to attend to the impact of these systems on human populations—and certainly not to the influences on our most vulnerable individuals and communities worldwide. Our educational institutions are in a race to catch up to the reality that is already knocking at our doors but has been predicted for decades.

In 1965, British mathematician I. J. Good wrote, An ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind.¹ As we enter an age in which companies like Uber, Argo, and Aurora are testing driverless cars in Pittsburgh and innovative interfaces like IBM’s Watson can play Jeopardy! and learn techniques for medical diagnoses, we are negotiating both how we navigate this intelligence explosion and how these explorations influence our understanding and perhaps preservation of what it means to be human. The future of human-machine relations will likely define our historical epoch, and yet many young technologists and humanists underestimate the downstream impact of technological innovations on human society.² Our leading experts are also scrambling to prepare for a future that has begun before it was expected. We grasp at metaphors for language to describe these systems and what we expect they can or will do in an immediate or far-flung future. Popular reports betray a lag between the dreams associated with these systems’ ultimate functionality and their present levels of efficacy. Uber may describe a near future in which no one needs to drive and accidents reduce to naught, but we know that as of summer 2019, the systems are often timid—or worse, incapable of making the decisions needed to safely navigate populated roads in cities like Pittsburgh, San Francisco, or Tempe.³ To positively influence the future arc of AI and humanity, we must study human-machine relations in the context of our human past and our technology’s possible future. In this book, we offer that very practice to be repeated in dynamic ways by our readers: curious citizens, university students, or future technologists and inventors.

Notes

1. Good, Speculations, 33.

2. Gieryn, Cultural Boundaries of Science.

3. On a visit to Uber’s Pittsburgh headquarters, an engineer driver described Uber’s software roll-out in Tempe, Arizona, in which a pedestrian was hit in an accident.

Acknowledgments

This book would not exist but for the encouragement offered by Marie Lufkin Lee at MIT Press. Marie sat with us, listened to the story of our experimental class, and proposed that we convert our lectures into a book. Her sound advice and vision in regard to this teaching project provided the fuel that we needed to dedicate two years to this collection of essays. The students in our inaugural AI and Humanity seminars were also essential to this effort; their conversations, questions, and explorations were catalysts that helped this work to mature from ideas into the more carefully framed inquiries that you will find in this book.

Our readers provided invaluable guidance to take this book from a ragged, early concept piece to finished form: Professor James Knapp and Professor Peggy Knapp, Professor Michael Genesereth, Maureen Bard, Professor Colin MacCabe, and Steven Ketchpel offered close readings of early drafts. Amy Burke performed comprehensive image rights research; Mitra Nourbakhsh undertook early transcription work. Thanks to Sophie Smith for exhaustively transcribing nearly all of the AI and Humanity Oral Archive and to Sophie and Joon Jang for creating the course website at aiandhumanity.org. Brian Staszel elegantly compensated for our inadequacies as videographers during video postproduction. Our colleagues at the Dietrich College of Humanities and Social Sciences, including Dean Richard Scheines, Professor David Danks, and Associate Dean Brian Junker, offered conversations that informed our respective thinking. At the Robotics Institute and in the School of Computer Science, Professor Reid Simmons, Professor Henny Admoni, and the CREATE Lab offered help in sharpening the ideas captured in this book. Our respective colleges offered the time, support, and space to pilot our teaching efforts with students in our respective schools. Our thanks go to our colleagues in the dean’s offices of both schools—namely, Associate Dean Guy Blelloch, Assistant Dean Tom Cortina, and Dean Richard Scheines—for their support in this curricular experimentation.

We are thankful for Stephanie Cohen’s assistance in our production process at MIT. We are also thankful for Professor John Carson’s collegiality and contribution to cover art. We also thank Rita Duffy for her inspirational work. Finally, we thank our families for putting up with the time that we continue to invest in our respective academic enterprises.

1 Introduction

Figure 1.1

Swiss automaton writing letters longhand, circa 1770.

The immediate question that was in my mind is, is this different from when owners of stables started to lose their jobs as horses were replaced with cars.

—Andrew Moore, Google, former dean of the School of Computer Science, Carnegie Mellon University¹

But the question is, is it more or less than the death that would happen if it was just a person who was driving the car, right? … Is death more acceptable when a human being does it than when a machine does it?

—Louis Chude-Sokei, professor of English, chair of African American Studies, Boston University²

People talk a lot about oversight, humans having oversight over AI, but if you really go back and think about it, who’s making more mistakes, humans or AI? Well, I would argue it’s humans are making a lot more mistakes than AI, so maybe there should be AI oversight over humans and maybe it should be a little bit of both ways.

—Tuomas Sandholm, professor of computer science, Carnegie Mellon University³

I’m a great fan of driverless car technology, it’s just that we’re not doing it right.

—Alan Winfield, professor of robot ethics, University of the West of England⁴

This textbook offers an analytical framing and a developing language for identifying, describing, and attending to the influences of contemporary technological advancement on societal structures and customs. Our uncertain future will be shaped and influenced by the advancements of the coming two decades, a glimpse of which we can already see today. As we build skills to shape and ensure responsiveness to this developing future, this book offers techniques for building a common language to describe and analyze the future that is now unfolding. Together we will attend to questions pertaining to individual and group identity, the influence of technological innovation on society over time, and explorations of the possible areas in which the next generation of thinkers is likely to need clarity and confidence to safeguard our societies. Our readers, students, and communities will develop policies, laws, and expectations for participating in this near future, protecting the features of human society that we most value.

The diversity and dynamism of our readers, combined with the boundary content of this book, make semantics and language a foundational aspect of this shared work. Words such as autonomy, agency, technology, and identity are threatened with reformulation as artificial intelligence shifts our understanding of what it means to be human and what it means for a machine to behave in human-like ways. For humanists and technologists, these words may lack the comprehensive etymological context that can help to situate discussions regarding new technology. Just as importantly, precise language is a crucial tool to help identify parallels between new technological systems that shift or change user and machine relationships and old negotiations of power. This intertwining of a historical relationship with evolving technological tools, their influence on human relationships, and analyses of how these past examples equip us to navigate the unfolding future will be our joint exploration in AI and Humanity.

The structural backbone of this book is steeped in the etymology of critical terms, drawing attention to the circulation and evolution of the English language over time. Using Keywords by Raymond Williams and Keywords for Today by Colin MacCabe and Holly Yanacek as foundational texts, we explore the cultural and etymological roots of boundary words relevant to AI and society. These terms drive forward an ever-expanding common language that is used to evaluate historical and future-facing explorations of technology and humanity, ranging from plays like Karl Čapek’s RUR (Rossum’s Universal Robots) to memoirs like Frederick Douglass’s Narrative