Sign in to continue learning
Mathematics for Machine Learning and Data Science
Beginner
Topics
Deep Learning
Mathematical Foundations
Supervised Learning
Now it turns out that there is a much faster way to tell if a matrix is singular or non-singular, and in this video you will learn it. It's called the determinant, and it is a quick formula that returns a zero if the matrix is singular, and a number different from zero if the matrix is non-singular. Take a look back at the matrix in the previous example, and let's focus on the singular matrix on the right. In this matrix, you can multiply the first row by 2 to obtain the second row, and therefore the rows are linearly dependent. In contrast, for the non-singular matrix on the left, there's no number that you can multiply the first row by to get the second row, because the rows are linearly independent. Let us look at this more carefully. If a matrix has entries A, B, C, and D, then the matrix is singular if there exists a number k for which the first row times k is equal to the second row. That means A k equals C, and B k equals D for the same value of k. This is equivalent to C over A equals D over B equals k, and that's the same thing, forgetting about k, that A D is equal to B C, or, equivalently, A D minus B C equals zero. This value of A D minus B C is very important. We're going to call it the determinant of the matrix. So the determinant of the matrix is A D minus B C. And by construction, this determinant is zero if the matrix is singular, and non-zero if it's non-singular. Now, a way I like to look at the determinant is like this. A D is the product of the numbers in the main diagonal, and B C is the product of the numbers in the anti-diagonal. So let's calculate the determinants of these two matrices in the ongoing example and see what you get. The first matrix has determinant 1 times 2 minus 1 times 1, which is 1. And the second one, 1 times 2 minus 2 times 1, which is 0. So notice that the first matrix, which is non-singular, has a determinant of 1, which is non-zero. And the second matrix, which is singular, has a determinant of 0. This is no coincidence. This is always going to be the case. Non-singular matrices have non-zero determinants. They don't need to be 1, but they're non-zero. And singular matrices have zero determinants. To summarize, the determinant of a matrix with entries A, B, C, and D is A D minus B C. And it is 0 precisely when the matrix is singular, and non-zero when the matrix is non-singular. So now you're ready for a quiz. In problem 1, you have two matrices, and you're asked to find their determinants. And in problem 2, determine if the matrices are singular or non-singular based on the results of problem 1. And the answers are, for the first matrix, the product of the terms in the main diagonal is 5 times 3, which is 15. The product of the terms in the main antidiagonal is 1 times minus 1, which is minus 1. And the difference is 15 minus minus 1, which is 15 plus 1, which is 16. Since the determinant is different from 0, then the matrix is non-singular. For the second matrix, the determinant is 2 times 3 minus minus 1 times minus 6. So the 3 minuses mean that's 1 minus. So the determinant is 6 minus 6, which is precisely 0. Since the determinant is 0, then the matrix is singular. The determinant for a 3 by 3 matrix is a bit more complicated than that for a 2 by 2 matrix, but it is mostly the same. Here it is. Before calculating the determinant, recall that for a 2 by 2 matrix, you consider the main diagonals, add the products of the elements in one of them, and subtract the products of the elements in the other one. In a larger matrix, this is the same thing, except you need some more diagonals. The first diagonal is the main one over here. Now, consider the next diagonal here. This one is incomplete, so you can complete it by wrapping around the matrix and finishing it like this. The same thing happens with the next diagonal. So, for the determinant, you're simply going to add the products of the elements in these diagonals, and subtract the products of the elements in these diagonals that go the other way around. Let's do an example. Consider this 3 by 3 matrix over here. In order to calculate the determinant, consider the main diagonal with entries 1, 2, and 2. Its product is 1 times 2 times 2, which is 4. Now consider the next one with entries 1, 1, 1, so the product of these entries is 1. And now the next one, again, with product of entries 1. And now we're going to subtract the diagonals that go in the other direction. So this one is 1 times 2 times 1, which is 2. This one over here is 1 times 1 times 1, which is 1. And this one over here is 1 times 1 times 2, which is 2. So we're adding 4 plus 1 plus 1 minus 2 minus 1 minus 2, and the determinant becomes 1. So a little more calculations, but basically the same thing as with 2 by 2 matrices. So now you're ready for a quiz. Find the determinant of the following matrices. And verify that those with determinant 0 are precisely the singular matrices. You can recognize that these are the exact same matrices from the previous quiz. And here are the answers. The first one has determinant 0, and so does the second one. The third one has determinant 6, and the fourth one has determinant 0. Therefore, the first two are singular, the third is non-singular, and the fourth is singular. And the same thing happens as with 2 by 2 matrices. Singular matrices have determinant 0, and non-singular matrices have determinant non-0. Now let's look more carefully at the third matrix, the one with determinant 6. This 6 was obtained as 6 plus 0 plus 0 minus 0 minus 0 minus 0. Why? Well, let's do it. You first have the main diagonal, which is 1 times 2 times 3, which is 6. The next one is 0, the next one is 0, and the negative ones are 0, 0, and 0. Notice something peculiar here. The only term that is non-zero is the first one. Because this matrix is very special, it's upper triangular. It means that everything below the diagonal is a 0. And now, since everything below the diagonal is a 0, all the terms in the determinant are going to contain one of these elements below the diagonal, except for the one that takes the entire main diagonal. So whenever you have a matrix where anything underneath the diagonal is 0, the determinant is going to be the product of the elements in the main diagonal. So that's a quick shortcut you can use. And they can still be singular. For example, look at this one. In the same way, this determinant is going to be the product of 1 times 2 times 0, the elements in the diagonal, and that happens to be 0. All right, that's the last new concept this week. After this, you'll have a few different opportunities to deepen your understanding. There are two labs that will help you get familiar with NumPy, the Python package you'll be using on labs and assignments throughout this course. The first lab is an introduction to the NumPy library overall, and in particular, NumPy arrays. If you're already familiar with this library, feel free to just skim this first lab. The second lab introduces the ways NumPy can be used to represent systems of equations and show you how to use plotting libraries to visualize those systems. Both labs are ungraded, but provide valuable background for the rest of the course. Finally, the week concludes with a graded quiz on all the topics you've studied this week. Good luck!
