The Book Of Why - Summary

> Summary: The book of Why by Judea Pearl & Dana Mackenzie

Probabilities, Causation and Correlation - sounds familiar yet confusing!

Oh! I really do not get stuff like this especially statics.

But let me be clear, this book was written from a statical and philosophical perspective by well known computer scientist and philosopher Judea Peral, as a reader I found it a bit difficult to understand the things while reading this book but I stick with it somehow. Really it took a lot for me to continue this book but it was worth it. I get to know the things from a statical perspective and it blow my mind as their’s numbers and probabilities everywhere in our day to day life we are living. Numbers are everywhere, it is hidden part of our day to day life. Even our brain is using it in a way that we are unaware of.

Book has 10 chapter with data, inference, causation and correlation, lurking variable, smoker’s debate, paradox galore, mining words, mediation, questions about big data & AI.

1. Mind over data

   • Casual Inference is posits that the human brain is the most advanced tool ever devised for managing causes and effects by simply asking the question “WHY?”

   • Our brain stores very large amount of knowledge which was supplied by data surrounding us

   • The ability to express the information in mathematical forms gave us the direction to combine our knowledge with data to answer causal questions

   • “Correlation is not causation.”

   • Data is Dump: Data can tell you that the people who took a medicine recovered faster than those who did not take it, but they can’t tell you why

2. From Buccaneers to guinea pigs: The Genesis of Causal inference

   • Central Limit Theorem the miracle of 19th century mathematics

   • Galton board: visual demonstration of Laplace’s theorem

   • Pearson’s skepticism: the universe is a product of human thought and that science is only a description of those thoughts. Thus causation, construed as an objective process that happens in the world outside the human brain, could not have any scientific meaning

   • Sewall Wright: first person to develop a mathematical method for answering causal questions from data, known as path diagrams

   • Bayesian connection: Bayesian statistics give us an objective way of combining the observed evidence with our prior knowledge (or subjective belief) to obtain a revised belief and hence a revised prediction of the outcome of the coin’s next toss

3. From evidence to causes: Reverend Bayes meets Mr. holmes

   • “When you have eliminated the impossible, whatever remains, however improbable, must be the truth.”

   Baye’s rule:

   • (1) formulate a hypothesis

   • (2) deduce a testable consequence of the hypothesis

   • (3) perform an experiment and collect evidence

   • (4) update your belief in the hypothesis

4. Confounding and Deconfounding: Or slaying the lurking variable

   • Contribution of statistics to causal inference: the randomized controlled trial (RCT)

   • Biblical story: Daniel and his companions prospered on the vegetarian diet which profound the way to conduct experiments today

   • The causal diagrams make possible a shift of emphasis from confounders to deconfounders

5. Smoke filled debate: clearing the air

   • History’s one of the most important scientific arguments against the smoking-cancer hypothesis was the possible existence of unmeasured factors that cause both craving for nicotine and lung cancer

   • Abe and Y ak’s smoke-filled debates was neither tobacco nor cancer, but the word “caused”

   • Tobacco: A Manmade Epidemic

   • “The beauty of causal diagrams is that they make the source of bias obvious.”

6. Paradoxes galore

   • Causal paradoxes shine a spotlight onto patterns of intuitive causal reasoning that clash with the logic of probability and statistics

   • The perplexing Monty hall problem

   • The berkson’s paradox

   • The Simpson’s paradox

7. Beyond adjustment: The conquest of Mount Intervention

   • Second level of the ladder of Causation: the level of intervention

   • Back door adjustment formula & Front door criterion

   • Case of Dr. John Snow: skeptical of the miasma theory

   • What is “good” and “bad” cholesterol?

8. Counterfactuals: Mining the words that could have been

   • Responsibility and blame, regret and credit: these concepts are the currency of a causal mind. To make any sense of them, we must be able to compare what did happen with what would have happened under some alternative hypothesis

   • How the language of counterfactuals can capture this elusive notion and how to estimate the probability that a defendant is culpable

9. Mediation: Search for mechanism

   • The search for mechanisms is critical to science, as well as to everyday life, because different mechanisms call for different actions when circumstances change

   • Direct effects (which do not pass through a mediator) and Indirect effects (which do)

   • In search of language: The Berkeley admission paradox

   • Mediation in linear wonderland

10. The Big data, AI & Big question

    • Author’s thought on Big data & AI

    • How do we extract meaning from all these numbers, bits, and pixels?

    • In technical terms, machine-learning methods today provide us with an efficient way of going from finite sample estimates to probability distributions, and we still need to get from distributions to cause-effect relations

    • AI and free will: understanding the benefits of the illusion of free will is the key to the stubbornly enigmatic problem of reconciling it with determinism

My thoughts on reading this book:

Judea Pearl is well known computer scientist and philosopher for developing theory of causal and counterfactual inference based on structural models. Pearl was awarded with Turing Award for his fundamental contributions to artificial intelligence through the development of a calculus for probabilistic and causal reasoning. I am glad to read such an amazing book with well defined theories and insights.

This book teaches me the importance of asking the “WHY” and in the quest of finding answer to the question, you got to learn many things and eventually it leads you to the answer you are looking for. I agree to the pearls opinion on to mention the failures, as history never mentions the struggles or failures of unsuccessful experiments but they are also very important to get the right direction. Failures are essential steps for learning and growing.

Things I learned:

• Data is not everything, it is how we interpret it

• Skepticism are important

• If you can express it in mathematical form, you are already half a way to go

• Visualization (diagrams, graphs) are more powerful than equations

• Historical debats can be server as learning

• Possibilities and Probablities are everywhere

• Observation is the key to find the answers

• Human Brain is the most powerful thing we ever had

• Causation: A quest for searching the answer to the “Why?”