Sign in to continue learning
Mathematics for Machine Learning and Data Science
Beginner
Topics
Deep Learning
Mathematical Foundations
Supervised Learning
Welcome back! You've seen a lot of fascinating concepts of probability so far. In this lesson, we will take a look at the concept of conditional probability. Conditional probability is all about calculating the probability of an event happening, given that another event has already happened. For example, let's say you are wondering about the probability that today is humid. That is some number. However, if you found out that yesterday was raining, the probability that today is humid changes. That is a conditional probability. So in a previous lesson, we established the probability of obtaining two heads when you throw two coins as follows. Let me recap very quickly. The first head can be heads or tails, and for each option of these two, the second one can be heads or tails. So we have four possibilities, heads-heads, head-tails, tail-heads, and tails-tails. And the probability of heads-heads? Well, the sample space was of size 4, and the event was of size 1, so it was 1 over 4. Now let me ask you a question. What is the probability that both coins land on heads, if you know that the first one landed in heads? So now you have more information. I came and told you that the first coin landed in heads. Now, what is the probability that both of them landed in heads? Well, if the first one landed in heads, then we leave here. This is the new sample space. The bottom part doesn't count anymore, because the first coin landed in heads, not tails. So now the denominator is not 4, but it's 2, because now there's two cases instead of four, and we still have one favorable outcome. So the new answer is 1 over 2. This over here is called a conditional probability, and we denote it with a vertical bar. So the probability of heads-heads, given that the first is heads, that given that, is denoted by a vertical bar, and this is a conditional probability. This can be seen in a table as follows. If the first coin is on the left and the second coin is on top, then these are the four possibilities. Heads-heads, heads-tails, tails-heads, and tails-tails. And we're going to look at the probability of landing on heads twice, given that the first one is heads. So we live in this first row, and when we live in the first row, our sample space now has size 2, and our event has size 1. Therefore, the probability is not one quarter anymore, it's now one half. Now, let me ask you another question. What is the probability of landing in heads twice, given that the first coin landed in tails? So slightly different question, give it a try. Well, if the first coin landed in tails, now we live here. This is our sample space, and the top is not valid anymore, so the denominator is now 2, but the numerator is 0, because now there's no heads-heads. So the answer is 0 over 2, which is 0. And this makes sense, right? Because if one of them landed in tails, there's no way that they're both going to land in heads. So this condition really changed the probability from one quarter, all the way to 0. And again, we can see it as a table, where we have the four cases, and we want to calculate the probability of landing in heads twice, given that the first one is tails, means we live in the second row, not in the first one. And the probability here is now, well, on the bottom we have a sample space of size 2, but on the top we have nothing, because there's no heads-heads in this sample space, and therefore the number is 0. So as you can see, a condition can change the probability of an event, and this is what we study with conditional probability. So let's recall the product rule for independent events. It said that the probability of A intersection B is P of A times P of B, but this only happened when A and B were independent. That doesn't always happen. Let me give you an example of something where the events are not independent. Let me ask you a question. What is the probability, if I roll two dice, that the first one lands on 6, and that the sum is 10? Well, we have a space of 36 possibilities, because it's pairs of numbers, and the only favorable one is 6-4, because if the first one is 6, and the sum is 10, the second one has to be 4. So the probability is 1 divided by 36 total ones, and that's 1 over 36. Now, let's look at it in a slightly different way. Let's try to apply the product rule. So we want the probability that the first one is 6, and that the sum is 10, and we would like to have the probability of the first one is 6. Well, that is over here, so that is 6 divided by 36. Now, what do I have to multiply this to obtain the probability that the first is 6 and the sum is 10? Well, I need the probability that the sum is 10, but among the ones for which the first is 6. So that is this event over here, the 6-4, divided by the 6 where the first one is 6, and that is 1 over 36, which is what we obtained here previously. So to summarize, we have that the probability of A intersection B, that's that the first is 6 and that the sum is 10, is the probability of the first one, that the first is 6, times the probability of the second one, given the first one. We need that given the first one, because we need to multiply by the probability of the sum is 10, given that the first is 6. So we get a slightly different formula. Probability of A intersection B is the probability of A times the probability of B given A. Now, when they're independent, then probability of B given A is the same as probability of B, because A doesn't make any difference on the occurrence of B. So when they're independent, it actually translates to P of A times P of B. But in the general case, it's P of A times P of B given A. And that is the general product rule that works every time. That means when the events are independent and when they're not independent. Now let's look at a dice example. Here's a question that you've seen before. What is the probability of the sum of two dice that I throw randomly is 10? Well, there's three cases divided by the total 36 of them. So the answer is 3 divided by 36, which in lowest terms, it's 1 over 12. Now let's put a condition. I'm telling you that I looked at the first one and the first one is a 6. So what is the probability that now the sum is 10, given that the first one is 6? Well, let's calculate it. Because the first one is 6, now we live on this bottom row. The top five rows don't matter anymore, because our new sample space is just this bottom row. And among this bottom row, well, there's six cases and one of them is favorable. So the probability is now 1 sixth. So the probability changed from 1 twelfth to 1 sixth. Now let's look at another problem. What is the probability that the sum is 10, but now the condition is that the first one was a 1. So I looked and the first one was a 1. Now what is the probability that the sum of them is 10? Well, if the first one is a 1, then now we live on this row over here, the first one. And on this row, there's six elements and none of them satisfy that the sum is 10, because the ones where the sum is 10 are on the very bottom. Therefore, the probability is 0. And it makes sense, right? If the first one is a 1, there's no way I can add the 10, because the second one can at most be a 6. So this probability changed to 0.
