Sign in to continue learning
Deep Learning Specialization
Intermediate
Topics
Computer Vision
Deep Learning
NLP
Supervised Learning
Transformers
In this video, I want to show you a slightly more complex example, where the slope of the function can be different at different points in the function. Let's start with an example. Here I've plotted the function f of a equals a squared. Let's again look at the point a equals 2. So a squared, or f of a, is equal to 4. Let's nudge a slightly to the right, so now a is equal to 2.001. f of a, which is a squared, is going to be approximately 4.004. It turns out that the exact value, if you plot and calculate and figure this out, is actually 4.004001. But I'm just going to say 4.004 is close enough. So what this means is that when a is equal to 2, let's draw this on the plot. So what we're saying is that if a is equal to 2, then f of a is equal to 4. And here the x and y axes are not drawn to scale. Technically, this vertical height should be much higher than this horizontal height, so the x and y axes are not on the same scale. But if I now nudge a to 2.001, then f of a becomes roughly 4.004. So if you draw this little triangle again, what this means is that if I nudge a to the right by 0.001, f of a goes up 4 times as much by 0.004. So in the language of calculus, we say that the slope, that is the derivative of f of a, at a equals 2 is 4. Or to write this out with our calculus notation, we say that dda of f of a is equal to 4 when a is equal to 2. Now, one thing about this function, f of a equals a squared, is that the slope is different for different values of a. This is different than the example we saw on the previous slide. So let's look at a different point. If a is equal to 5, so instead of a equals 2, we now have a equals 5. Then a squared is equal to 25, so that's f of a. If I nudge a to the right again, it's a tiny little nudge to a. So now a is 5.001, then f of a will be approximately 25.010. So what we see is that by nudging a up by 0.001, f of a goes up 10 times as much. So we have that dda f of a is equal to 10 when a is equal to 5, because f of a goes up 10 times as much as a does when I make a tiny little nudge to a. So one way to see why the derivative is different at different points is that if you draw that little triangle at different locations on this, you see that the ratio of the height of the triangle over the width of the triangle is very different at different points on the curve. So here, the slope is equal to 4 when a is equal to 2, but is equal to 10 when a is equal to 5. Now, if you pull up a calculus textbook, a calculus textbook will tell you that dda of f of a, so f of a is equal to a squared, so that's dda of a squared. One of the formulas you find in the calculus textbook is that this thing, the slope of the function a squared, is equal to 2a. I'm not going to prove this, but the way you find this out is that you open up a calculus textbook to the table of formulas, and it'll tell you the derivative of 2 of a squared is equal to 2a. And indeed, this is consistent with what we've worked out. Namely, when a is equal to 2, the slope of the function 2a is 2 times 2, is therefore equal to 4. And when a is equal to 5, then the slope of the function 2 times a is 2 times 5, is equal to 10. So if you ever pull up a calculus textbook and you see this formula, that the derivative of a squared is equal to 2a, all that means is that for any given value of a, if you nudge it upward by 0.001, already a tiny little value, you would expect f of a to go up by 2a. That is the slope or the derivative times however much you had nudged to the right the value of a. Now, one tiny little detail, you know, I had used these approximate symbols here, and this wasn't exactly 4.004. There's an extra 0.001 hanging out there. It turns out that this extra 0.001, this little thing here, is because we were nudging a to the right by 0.001. If we're instead nudging it to the right by this infinitesimally small value, then this extra error term will go away. And you find that the amount that f of a goes up is exactly equal to the derivative times the amount that you nudge a to the right. And the reason why it's not 4.004 exactly is because derivatives are defined using these infinitesimally small nudges to a, rather than, you know, 0.001, and while 0.001 is small, it's not infinitesimally small. So that's why the amount that f of a went up isn't exactly given by the formula, but is only kind of approximately given by the derivative. To wrap up this video, let's just go through a few more quick examples. The example you've already seen is that if f of a equals a squared, then the calculus textbook's formula table will tell you that the derivative is equal to 2a. And so the example we went through was that if a is equal to 2, f of a equals 4. And if we nudge a so it's a little bit bigger, then f of a is about 4.004. And so f of a went up four times as much. And indeed, when a is equal to 2, the derivative is equal to 4. Let's look at some other examples. Let's say instead that f of a is equal to a cubed. If you go to a calculus textbook and look up the table of formulas, you see that the slope of this function, again the derivative of this function, is equal to 3a squared. So you can, you know, get this formula out of the calculus textbook. So what this means, so the way to interpret this is as follows. Let's take a equals 2 as an example again. So f of a or a cubed is equal to 8. That's 2 to the power of 3. So if we give a a tiny little nudge, you find that f of a is about 8.012. And feel free to check this. Take 2.001 to the power of 3. You find that it's very close to 8.012. And indeed, when a is equal to 2, that's 3 times 2 squared. That's equal to 3 times 4, which is equal to 12. So the derivative formula predicts that if you nudge a to the right by a tiny little bit, f of a should go up 12 times as much. And indeed, this is true. When a went up by .001, f of a went up 12 times as much by .012. Just one last example and then we'll wrap up. Let's say that f of a is equal to the log function. Right, so I'm going to write log of a. I'm going to use this as the base e logarithm. So some people write that as log lon of a. So if you go to a calculus textbook, you find that when you take the derivative of log of a, so this is a function that just looks like that. The slope of this function is given by 1 over a. So the way to interpret this is that if a is any value, again, let's just keep using a equals 2 as an example, and you nudge a to the right by .001, you would expect f of a to go up by 1 over a, that is by the derivative, times the amount that you increased a. So in fact, if you pull up a calculator, you find that if a is equal to 2, f of a is about 0.69315, and if you increase a to 2.001, then f of a is about 0.69365, so it's gone up by 0.0005. And indeed, if you look at the formula for the derivative, when a is equal to 2, d d a f of a is equal to 1 half. So this derivative formula predicts that if you bump up a by .001, you would expect f of a to go up by only 1 half as much, and 1 half of .001 is 0.0005, which is exactly what we got, right? That when a goes up by .001, going from a equals 2 to a equals 2.001, f of a goes up by half as much, so it ends up going up by approximately .0005. So if you draw that little triangle, if you will, is that if on the horizontal axis, this goes up by .001, on the vertical axis, log of a goes up by half of that, so .0005. And so that 1 over a, or 1 half in this case, that's just the slope of this line when a is equal to 2. So that's it for derivatives. There are just two take-home messages from this video. First is that the derivative of a function just means the slope of a function, and the slope of a function can be different at different points on the function. In our first example, where f of a equals 3a, that was a straight line, the derivative was the same everywhere. But for other functions, like f of a equals a squared, or f of a equals log of a, the slope of the line varies, and so the slope or the derivative can be different at different points on the curve. So that's the first takeaway. Derivative just means slope of a line. Second takeaway is that if you want to look up the derivative of a function, you can flip open, you know, calculus, or textbook, or look at Wikipedia, and often get a formula for the slope of these functions at different points. So with that, I hope you have an intuitive understanding of derivatives or slopes of lines. Let's go on to the next video. We'll start to talk about the computation graph and how to use that to compute derivatives of more complex functions.
