JK not jk @OneByteMore

Every Data Scientist needs to know these ideas. They will blow your mind. 1. Correlation vs Causation P(A | B) is the probability of A given B. It is the probability that we will observe A given that we have already observed B. P(A | do(B)) is the probability of A given do(B). It is the probability that we will observe A given that we have intervened to cause B to happen. In this context, an intervention simply means to take an action of some kind. Therefore do(B) means to take an action which causes B to happen. The expressions P(A | B) and P(A | do(B)) might seem very similar but they represent very different situations. 2. We can only learn P(A|B) from the data alone. Bob has an extremely accurate weather app and is always very good about bringing his umbrella when it rains. We observe Bob over several years and we find that whenever it rains, Bob always has his umbrella and he never brings his umbrellas on days when it doesn't rain. In the language of probability, we say P(Umbrella | Rain) = 1 and P(Rain | Umbrella) = 1 as well. What we can learn from this data alone is how to predict whether it rains with a 100% accuracy by checking whether Bob has an umbrella. We can also learn to predict with 100% accuracy whether Bob has an umbrella by checking if it's going to rain. What we cannot learn is what will happen if we give Bob an umbrella on a random day of our choosing. The answer to this question is P(Rain | do(Umbrella) ) and it's unknowable from the data alone. We need prior knowledge about how the world works to properly interpret the data we collected. We need to know that rain has an effect on Bob's behavior, but Bob's behavior has no effect on the rain. Information about the effects of interventions are simply not available in raw data unless it is collected by controlled experimental manipulation. 3. Scientific Experiments work because they produce a very special kind of data. You may have heard of what many people call a scientific experiment. Take a collection of objects, animals or people. Randomly split that collection into a control group and a treatment group. Apply your intervention to the treatment group while leaving the control group alone. If you observe any differences between the treatment group and the control group, it is logical to attribute these differences to the treatment. You can therefore say the differences were caused by the treatment. In statistics, the procedure I just described is called a Randomized Controlled Trial. It is a procedure for generating a specific kind of data where: P(Difference | Treatment) = P(Difference | do(Treatment) ) This is why traditional science experiments work. They are designed to capture causal information. This is not the case for vast majority of data that we collect in society. Without human guidance or access to real world knowledge, statistical algorithms and artificial intelligences can only learn P(A | B) from the raw data. This is a fundamental mathematical limitation on the use of data alone. That's it for now. This post is part of a series of posts about the concept of causal inference. They are based on the content of the Book of Why by Judea Pearl with lots of commentary from me. Follow me (@kareem_carr) so you don't miss out on the next post. Please show support by liking and retweeting the thread.

This seems really interesting

Pavel A. Samsonov @PavelASamsonov

4 years ago

In output-first planning, present is clearer than future: we only know roughly what features we will build in Q4 but here are our Q1 user stories In outcome-first planning, future is clearer than present: our Q4 goals are set, in Q1 we will look for the right paths to reach them

10 69 471 0 289

@DisruptedVp If you can't find customers who are delighted with the product and want to make it better (out of self interest—to get changes that they themselves want), I'd wonder if you're building the right product.

Research in AI is surprisingly more accessible to people with different backgrounds compared to other fields. Anyone (w/ relevant experience) can contribute to impactful research. Here are 5 research orgs you can join to contribute to real, open research in deep learning ↓

@tottinge My wife and I have been enjoying watching Haven. The quiet emphasis on restorative justice is a refreshing change of pace.

For anyone who thinks COVID is going to go away on its own:

T. Ryan Gregory 🇨🇦 @TRyanGregory

4 years ago

Sigh. 🧵 * Viruses don't automatically or inevitably evolve to become milder. Transmissibility and immune escape can be under strong selection but virulence may not be, especially if there are lots of hosts, transmission is easy, and infectiousness precedes symptoms.

47 1K 4K 0 432

HOW MANY SHOTS WILL WE NEED TO GET?

@kathryniveyy Done

@Crave_PR @CraveCanada @billharris_tv @tylerjbjohnston @evanjstern @thelede_ca Rips!

@NehemiahKiv I only code on pirate ships, because I like to use R

@shanselman @james_output In this case, F = m1a1 = m2a2. The object with the smaller mass will accelerate faster. That could be thought of as getting pulled harder.

Can you describe your favorite movie as boring as possible?

I’m just sayin’

This thread is worth a read:

Erynn Brook @ErynnBrook

7 years ago

I’m waiting on kitty ultrasound results and trying to distract myself a little bit so I’d like to tell you a story about something that happened last night, in the hopes that I can process my feelings around it. I met a girl on the train last night.

1K 16K 43K 0 1K

@cjane87 43. Roasted with shallots

@TheRShafe @dtuite Plus one! Tickets lose value over time and should be treated as streaming data, not long term persistence.

Was really sad to hear this. Condolences to his family: google.ca/amp/s/www.cnbc…

If you only do what you want to do, you'll never be who want to be

Yet another reason why password managers make so much sense: I *never* sign up to a service using social login, I create a new account, generate the password via @1Password then store it there. Think of it as sand-boxing all your identities.

WIRED @WIRED

8 years ago

Facebook's data breach just got substantially worse than initially reported. In a follow up call this afternoon, the platform revealed that if your account was impacted, a hacker could have accessed any account you log into using Facebook wired.trib.al/yESjAaS

101 2K 1K 0 34