In the last video, you saw how to implement forward prop in Python, but by hard-coding lines of code for every single neuron. Let's now take a look at the more general implementation of forward prop in Python. Similar to the previous video, my goal in this video is to show you the code so that when you see it again in the practice lab and the optional labs, you know how to interpret it. So as we walk through this example, don't worry about taking notes on every single line of code. If you can read through the code and understand it, that's definitely enough. So what you can do is write a function to implement a dense layer that is a single layer of a neural network. So I'm going to define the dense function, which takes as input the activation from the previous layer, as well as the parameters w and b for the neurons in a given layer. Using the example from the previous video, if layer 1 has 3 neurons, and if w1 and w2 and w3 are these, then what we'll do is stack all of these weight vectors into a matrix. This is going to be a 2 by 3 matrix, where the first column is the parameter w11, the second column is the parameter w12, and the third column is the parameter w13. And then in a similar way, if you have parameters b, b11 equals negative 1, b12 equals 1, and so on, then we're going to stack these three numbers into one D-array, b, as follows, negative 1, 1, 2. So what the dense function will do is take as input the activation from the previous layer, and a here could be a0, which is equal to x, or the activation from a later layer, as well as the w parameters stacked in columns like shown on the right, as well as the b parameters also stacked into a one D-array like shown to the left, over there. And what this function will do is input a, the activation from the previous layer, and will output the activations from the current layer. So let's step through the code for doing this. Here's the code. First, units equals w dot shape 1, so w here is a 2 by 3 matrix, and so the number of columns is 3, that's equal to the number of units in this layer. So here, units would be equal to 3, and looking at the shape of w, it's just a way of pulling out the number of hidden units, or the number of units in this layer. Next, we set a to be an array of zeros with as many elements as there are units. So in this example, we need to output three activation values, so this just initializes a to be 0, 0, 0, an array of three zeros. Next, we go through a for loop to compute the first, second, and third elements of a. So for j in range units, so j goes from 0 to units minus 1, so it goes from 0, 1, 2, indexing from 0 in Python as usual. This command, w equals capital W colon comma j, this is how you pull out the jth column of a matrix in Python. So the first time through this loop, this will pull out the first column of w, and so it will pull out w11. The second time through this loop, when you're computing the activation of the second unit, it will pull out the second column corresponding to w12, and so on for the third time through this loop. And then you compute z using the usual formula as a dot product between that parameter w and the activation that you had received, plus bj. And then you compute the activation, aj equals g, sigmoid function applied to z. So three times through this loop, and you've computed the values for all three values of this vector of activations a, and then finally you return a. So what the dense function does is it inputs the activations from the previous layer, and given the parameters for the current layer, it returns the activations for the next layer. So given the dense function, here's how you can string together a few dense layers sequentially in order to implement forward prop in the neural network. Given the input features x, you can then compute the activations a1 to be a1 equals dense of x, w1, b1, where here w1, b1 are the parameters, sometimes also called the weights, of the first hidden layer. Then you can compute a2 as dense of a1, which you just computed above, and w2, b2, which are the parameters or weights of this second hidden layer, and then compute a3 and a4. And if this is a neural network with four layers, then the final output, f of x, is just equal to a4, and so you return f of x. Notice that here I'm using a capital W because one of the notational conventions from linear algebra is to use uppercase or capital alphabets when it's referring to a matrix, and lowercase to refer to vectors and scalars. So because this is a matrix, this is capital W. So that's it. You now know how to implement forward prop yourself from scratch. And you get to see all this code and run it and practice it yourself in the practice lab coming after this as well. I think that even when you're using powerful libraries like TensorFlow, it's helpful to know how it works under the hood. Because in case something goes wrong, in case something runs really slowly, or you have a strange result, or it looks like there's a bug, your ability to understand what's actually going on will make you much more effective when debugging your code. When I run machine learning algorithms a lot of the time, frankly, it doesn't work, certainly not the first time. And so I find that my ability to debug my code, be it TensorFlow code or something else, is really important to being an effective machine learning engineer. So even when you're using TensorFlow or some other framework, I hope that you find this deeper understanding useful for your own applications and for debugging your own machine learning algorithms as well. So that's it. That's the last required video of this week with code in it. In the next video, I'd like to dive into what I think is a fun and fascinating topic, which is what is the relationship between neural networks and AI or AGI, artificial general intelligence? This is a controversial topic, but because it's been so widely discussed, I want to share with you some thoughts on this. So when you are asked, are neural networks at all on the path to human level intelligence, you have a framework for thinking about that question. Let's go take a look at that fun topic, I think, in the next video.