Sign in to continue learning
Deep Learning Specialization
Intermediate
Topics
Computer Vision
Deep Learning
NLP
Supervised Learning
Transformers
Hello and welcome back. In this week, we're going to go over the basics of neural network programming. It turns out that when you implement a neural network, there are some implementational techniques that are going to be really important. For example, if you have a training set of m training examples, you might be used to processing the training set by having a for loop step through your m training examples. But it turns out that when you implement a neural network, you usually want to process your entire training set without using an explicit for loop to loop over your entire training set. So you'll see how to do that in this week's materials. Another idea, when you organize the computation of a neural network, usually you have what's called a forward pass or forward propagation step, followed by a backward pass or what's called a backward propagation step. And so in this week's materials, you also get an intuition about why the computations in learning in a neural network can be organized in this forward propagation and a separate backward propagation. For this week's materials, I want to convey these ideas using logistic regression in order to make the ideas easier to understand. But even if you've seen logistic regression before, I think that there will be some new and interesting ideas for you to pick up in this week's materials. So with that, let's get started. Logistic regression is an algorithm for binary classification. So let's start by setting up the problem. Here's an example of a binary classification problem. You might have an input of an image like that and want to output a label to recognize this image as being either a cat, in which case you output one, or non-cat, in which case you output zero. And we're going to use y to denote the output label. Let's look at how an image is represented in a computer. To store an image, your computer stores three separate matrices corresponding to the red, green, and blue color channels of this image. So if your input image is 64 pixels by 64 pixels, then you would have three 64 by 64 matrices corresponding to the red, green, and blue pixel intensity values for your image. Although to make these from a slide, I drew these as much smaller matrices. So these are actually 5 by 4 matrices rather than 64 by 64. So to turn these pixel intensity values into a feature vector, what we're going to do is unroll all of these pixel values into a input feature vector x. So to unroll all of these pixel intensity values into a feature vector, what we're going to do is define a feature vector x corresponding to this image as follows. We're just going to take all the pixel values, 2, 5, 5, 2, 3, 1, and so on, 2, 5, 5, 2, 3, 1, and so on until we've listed all the red pixels and then eventually 2, 5, 5, 1, 3, 4, 2, 5, 5, 1, 3, 4, and so on until we get a very long feature vector listing out all the red, green, and blue pixel intensity values of this image. So if this image is a 64 by 64 image, the total dimension of this vector x will be 64 by 64 by 3 because that's the total numbers we have in all of these matrices, which in this case turns out to be 1, 2, 2, 8, 8. That's what you get if you multiply out those numbers. And so we're going to use nx equals 1, 2, 2, 8, 8 to represent the dimension of the input features x. And sometimes for brevity, I might also just use lowercase n to represent the dimension of this input feature vector. So in binary classification, our goal is to learn a classifier that can input an image represented by its feature vector x and predict whether the corresponding label y is 1 or 0. That is whether this is a capped image or a non-capped image. Let's now lay out some of the notation that we'll use throughout the rest of this course. A single training example is represented by a pair x, y where x is an nx dimensional feature vector and y, the label, is either 0 or 1. Your training set will comprise lowercase m training examples, and so your training sets will be written x1, y1, which is the input and output for your first training example, x2, y2, for your second training example, up to xm, ym, which is your last training example, and then that altogether is your entire training set. So I'm going to use lowercase m to denote the number of training examples, and sometimes to emphasize that this is the number of training examples, I might write this as m subscript train, and when we talk about the test set, we might sometimes use m subscript test to denote the number of test examples. So that's the number of test examples. Finally, to put all of the training examples into a more compact notation, we're going to define a matrix, capital X, as defined by taking your training set inputs, x1, x2, and so on, and stacking them in columns. So you take x1 and put that as the first column of this matrix, x2, put that as the second column, and so on, down to xm, then this is the matrix, capital X. So this matrix X will have m columns, where m is the number of training examples, and the number of rows or the height of this matrix is nx. Notice that in other clauses, you might see the matrix, capital X, defined by stacking up the training examples in rows, like so, x1 transpose, down to xm transpose, but it turns out that when you're implementing neural networks, using this convention I have on the left will make the implementation much easier. So just to recap, X is a nx by m dimensional matrix, and when you implement this in Python, you see that x dot shape, that's the Python command for finding the shape of a matrix, that this is an nx comma m, this just means it's an nx by m dimensional matrix. So that's how you group the training examples, input X into the matrix. How about the output labels Y? It turns out that to make your implementation of a neural network easier, it would be convenient to also stack Y in columns. So we're going to define capital Y to be equal to Y1, Y2, up to Ym, like so. So Y here will be a 1 by m dimensional matrix, and again, to use the Python notation, you know, the shape of Y will be 1 comma m, which just means this is a 1 by m matrix. And as you implement your neural network later in this course, you find that a useful convention will be to take the data associated with different training examples, and by data, I mean either X or Y or other quantities you see later, but to take the stuff or the data associated with different training examples and to stack them in different columns, like we've done here for both X and Y. So that's the notation we'll use for logistic regression and for neural networks later in this course. If you ever forget what a piece of notation means, like what is m or what is n or what is something else, we've also posted on the course website a notation guide that you can use to quickly look up what any particular piece of notation means. So with that, let's go on to the next video where we'll start to flesh out logistic regression using this notation.
