2
1
3
2
Copyright
Copyright © 2018 by Judea Pearl and Dana Mackenzie
Hachette Book Group supports the right to free expression and the value of copyright.
The purpose of copyright is to encourage writers and artists to produce the creative works
that enrich our culture.
The scanning, uploading, and distribution of this book without permission is a theft of the
author’s intellectual property. If you would like permission to use material from the book
(other than for review purposes), please contact permissions@hbgusa.com. Thank you for
your support of the author’s rights.
Basic Books
Hachette Book Group
1290 Avenue of the Americas, New York, NY 10104
www.basicbooks.com
First Edition: May 2018
Published by Basic Books, an imprint of Perseus Books, LLC, a subsidiary of Hachette
Book Group, Inc. The Basic Books name and logo is a trademark of the Hachette Book
Group.
The Hachette Speakers Bureau provides a wide range of authors for speaking events. To
find out more, go to www.hachettespeakersbureau.com or call (866) 376-6591.
The publisher is not responsible for websites (or their content) that are not owned by the
publisher.
Library of Congress Cataloging-in-Publication Data
Names: Pearl, Judea, author. | Mackenzie, Dana, author.
Title: The book of why : the new science of cause and effect / Judea Pearl and Dana
Description: New York : Basic Books, [2018] | Includes bibliographical references and
Mackenzie.
index.
Identifiers: LCCN 2017056458 (print) | LCCN 2018005510 (ebook) | ISBN 9780465097616
(ebook) | ISBN 9780465097609 (hardcover) | ISBN 046509760X (hardcover) | ISBN
0465097618 (ebook)
Subjects: LCSH: Causation. | Inference.
4
3
Classification: LCC Q175.32.C38 (ebook) | LCC Q175.32.C38 P43 2018 (print) | DDC 501—
dc23
LC record available at https://lccn.loc.gov/2017056458
ISBNs: 978-0-465-09760-9 (hardcover); 978-0-465-09761-6 (ebook)
E3-20180417-JV-PC
5
4
CONTENTS
Cover
Title Page
Copyright
Dedication
Preface
INTRODUCTION Mind over Data
CHAPTER 1 The Ladder of Causation
CHAPTER 2 From Buccaneers to Guinea Pigs: The Genesis of Causal Inference
CHAPTER 3 From Evidence to Causes: Reverend Bayes Meets Mr. Holmes
CHAPTER 4 Confounding and Deconfounding: Or, Slaying the Lurking Variable
CHAPTER 5 The Smoke-Filled Debate: Clearing the Air
CHAPTER 6 Paradoxes Galore!
CHAPTER 7 Beyond Adjustment: The Conquest of Mount Intervention
CHAPTER 8 Counterfactuals: Mining Worlds That Could Have Been
CHAPTER 9 Mediation: The Search for a Mechanism
CHAPTER 10 Big Data, Artificial Intelligence, and the Big Questions
Acknowledgments
About the Authors
Also by Judea Pearl
Notes
Bibliography
Index
6
5
PREFACE
ALMOST two decades ago, when I wrote the preface to my book Causality (2000), I
made a rather daring remark that friends advised me to tone down. “Causality has
undergone a major transformation,” I wrote, “from a concept shrouded in mystery into a
mathematical object with well-defined semantics and well-founded logic. Paradoxes and
controversies have been resolved, slippery concepts have been explicated, and practical
problems relying on causal information that long were regarded as either metaphysical or
unmanageable can now be solved using elementary mathematics. Put simply, causality
has been mathematized.”
Reading this passage today, I feel I was somewhat shortsighted. What I described as a
“transformation” turned out to be a “revolution” that has changed the thinking in many of
the sciences. Many now call it “the Causal Revolution,” and the excitement that it has
generated in research circles is spilling over to education and applications. I believe the
time is ripe to share it with a broader audience.
This book strives to fulfill a three-pronged mission: first, to lay before you in
nonmathematical language the intellectual content of the Causal Revolution and how it is
affecting our lives as well as our future; second, to share with you some of the heroic
journeys, both successful and failed, that scientists have embarked on when confronted
by critical cause-effect questions.
Finally, returning the Causal Revolution to its womb in artificial intelligence, I aim to
describe to you how robots can be constructed that learn to communicate in our mother
tongue—the language of cause and effect. This new generation of robots should explain to
us why things happened, why they responded the way they did, and why nature operates
one way and not another. More ambitiously, they should also teach us about ourselves:
why our mind clicks the way it does and what it means to think rationally about cause and
effect, credit and regret, intent and responsibility.
When I write equations, I have a very clear idea of who my readers are. Not so when I
write for the general public—an entirely new adventure for me. Strange, but this new
experience has been one of the most rewarding educational trips of my life. The need to
shape ideas in your language, to guess your background, your questions, and your
reactions, did more to sharpen my understanding of causality than all the equations I have
written prior to writing this book.
For this I will forever be grateful to you. I hope you are as excited as I am to see the
results.
8
Judea Pearl
6
Los Angeles, October 2017
9
7
INTRODUCTION: MIND OVER DATA
Every science that has thriven has thriven upon its own symbols.
—AUGUSTUS DE MORGAN (1864)
THIS book tells the story of a science that has changed the way we distinguish facts from
fiction and yet has remained under the radar of the general public. The consequences of
the new science are already impacting crucial facets of our lives and have the potential to
affect more, from the development of new drugs to the control of economic policies, from
education and robotics to gun control and global warming. Remarkably, despite the
diversity and apparent incommensurability of these problem areas, the new science
embraces them all under a unified framework that was practically nonexistent two
decades ago.
The new science does not have a fancy name: I call it simply “causal inference,” as do
many of my colleagues. Nor is it particularly high-tech. The ideal technology that causal
inference strives to emulate resides within our own minds. Some tens of thousands of
years ago, humans began to realize that certain things cause other things and that
tinkering with the former can change the latter. No other species grasps this, certainly not
to the extent that we do. From this discovery came organized societies, then towns and
cities, and eventually the science- and technology-based civilization we enjoy today. All
because we asked a simple question: Why?
Causal inference is all about taking this question seriously. It posits that the human
brain is the most advanced tool ever devised for managing causes and effects. Our brains
store an incredible amount of causal knowledge which, supplemented by data, we could
harness to answer some of the most pressing questions of our time. More ambitiously,
once we really understand the logic behind causal thinking, we could emulate it on
modern computers and create an “artificial scientist.” This smart robot would discover yet
unknown phenomena, find explanations to pending scientific dilemmas, design new
experiments, and continually extract more causal knowledge from the environment.
But before we can venture to speculate on such futuristic developments, it is important
to understand the achievements that causal inference has tallied thus far. We will explore
the way that it has transformed the thinking of scientists in almost every data-informed
discipline and how it is about to change our lives.
The new science addresses seemingly straightforward questions like these:
10
8