Sign in to continue learning
Machine Learning Specialization
Beginner
Topics
Anomaly Detection
Deep Learning
Machine Learning
Supervised Learning
Unsupervised Learning
In the last video, you saw one visualization of the cost function J of W or J of WB. Let's look at some further, richer visualizations so we can get an even better intuition about what the cost function is doing. Here is what we've seen so far. There's the model, the model's parameters W and B, the cost function J of W and B, as well as the goal of linear regression, which is to minimize the cost function J of W and B over parameters W and B. In the last video, we had temporarily set B to 0 in order to simplify the visualizations. But now, let's go back to the original model with both parameters W and B without setting B to be equal to 0. Same as last time, we want to get a visual understanding of the model function f of x, shown here on the left, and how it relates to the cost function J of WB, shown here on the right. Here's a training set of house sizes and prices. Let's say you pick one possible function of x, like this one. Here, I've set W to 0.06 and B to 50. So f of x is 0.06 times x plus 50. Note that this is not a particularly good model for this training set. It's actually a pretty bad model that seems to consistently underestimate housing prices. Given these values for W and B, let's look at what the cost function J of W and B may look like. Recall what we saw last time was when you had only W because we temporarily set B to 0 to simplify things. Back then, we had come up with a plot of the cost function that looked like this as a function of W only. So when we had only one parameter W, the cost function had this U-shaped curve, shaped a bit like a soup bowl. Mmm, that sounds delicious. Now, in this housing price example that we have on this slide, we have two parameters, W and B, and so the plot becomes a little more complex. It turns out that the cost function also has a similar shape, like a soup bowl, except in three dimensions instead of two. In fact, depending on your training set, the cost function will look something like this. To me, this looks like a soup bowl, maybe because I'm a little bit hungry. Or maybe to you, it looks like a curved dinner plate or a hammock. Actually, that sounds relaxing too, and there's your coconut drink. Maybe when you're done with this course, you should treat yourself to a vacation and relax in a hammock like this. What you see here is a 3D surface plot where the axes are labeled W and B. So as you vary W and B, which are the two parameters of the model, you get different values for the cost function J of W and B. This is a lot like the U-shaped curve you saw in the last video, except instead of having one parameter, W, as input into J, you now have two parameters, W and B, as inputs into this soup bowl or this hammock-shaped function J. And I just want to point out that any single point on this surface represents a particular choice of W and B. For example, if W was minus 10 and B was minus 15, then the height of the surface above this point is the value of J when W is minus 10 and B is minus 15. Now, in order to look even more closely at specific points, there's another way of plotting the cost function J that would be useful for visualization, which is, rather than using these 3D surface plots, I'd like to take this exact same function J, so I'm not changing the function J at all, and plot it using something called a contour plot. And if you've ever seen a topographical map showing how high different mountains are, the contours in a topographical map are basically horizontal slices of the landscape of, say, a mountain. This image is of Mount Fuji in Japan. I still remember my family visiting Mount Fuji when I was a teenager. It's a beautiful sight. And if you fly directly above the mountain, that's what this contour map looks like. It shows all the points that are at the same height for different heights. At the bottom of this slide is the 3D surface plot of the cost function J. I know it doesn't look very bow-shaped, but it is actually a bow just very stretched out, which is why it looks like that. In an optional lab that is surely to follow, you'll be able to see this in 3D and spin around the surface yourself, and it'll look more obviously bow-shaped there. Next, here on the upper right is a contour plot of this exact same cost function as that shown at the bottom. The two axes on this contour plot are B on the vertical axis and W on the horizontal axis. What each of these ovals, also called ellipses, shows is the set of points on the 3D surface which are at the exact same height. In other words, the set of points which have the same value for the cost function J. To get a contour plot, you take the 3D surface at the bottom, and you use a knife to slice it horizontally. You take horizontal slices of that 3D surface and get all the points that are at the same height. Therefore, each horizontal slice ends up being shown as one of these ellipses, or one of these ovals. So concretely, if you take that point, and that point, and that point, all of these three points have the same value for the cost function J, even though they have different values for W and B. And in the figure on the upper left, you see also that these three points correspond to different functions, F, all three of which are actually pretty bad for predicting housing prices in this case. Now, the bottom of the boat, where the cost function J is at a minimum, is this point right here, at the center of these concentric ovals. If you haven't seen contour plots much before, I'd like you to imagine, if you will, that you're flying high up above the boat in an airplane or in a rocket ship, and you're looking straight down at it. So that is as if you set your computer monitor flat on your desk, facing up, and the boat shape is coming directly out of your screen, rising above your desk. Imagine that the boat shape grows out of your computer screen, lying flat, like that. So that each of these ovals have the same height above your screen, and the minimum of the boat is right down there, in the center of the smallest oval. So it turns out that the contour plots are a convenient way to visualize the 3D cost function J, but in a way that's plotted in just 2D. In this video, you saw how the 3D boat-shaped surface plot can also be visualized as a contour plot. Using this visualization tool, in the next video, let's visualize some specific choices of W and B in a linear regression model, so that we can see how these different choices affect the straight line you're fitting to the data. Let's go on to the next video.
Week 1: Introduction to Machine Learning
Certificate