Sign in to continue learning
Linear Algebra for Machine Learning and Data Science
34 hours 5 mins
The first topic you'll learn in linear algebra is systems of linear equations. However, before you delve into equations, it is important to understand the language of mathematics. When I think of equations, I think of sentences. Sentences that are giving you information about things in the world. And when there are many sentences, or systems of sentences as I call them, these sentences combine themselves to give you more information. When looked at from a proper angle, the way sentences combine to give you information is very similar to the way equations combine to give you information. In other words, systems of sentences behave a lot like systems of equations, as you'll see in the following example. So let's start with some examples of systems of sentences. Now, for the sake of this example, assume that you only have one dog and one cat, and they're both of only one color. You are given some information, and your goal is to try to figure out the color of each of the animals. So here's system 1 with the sentences, The dog is black and the cat is orange. Then you have system 2 with sentences, The dog is black and the dog is black. And finally, you have system 3 with the sentences, The dog is black and the dog is white. The sentences used here are simple sentences with one piece of information each. So sentences such as, The dog is black and the dog is white are not allowed, as they separately contain two pieces of information. And the goal for a system is to convey as much information as possible with these simple sentences. With regards to achieving that goal, notice that these systems are quite different. In particular, the first system of sentences contains two sentences and two pieces of information. This means the system contains as many pieces of information as sentences, and that's called a complete system. System 2 is a bit less informative, as it has two sentences, but they're exactly the same. Therefore, the system only carries one piece of information, even though it contains two sentences. The sentences here repeat themselves, and therefore the system is called redundant. And finally, system 3 is strange, as the sentences contradict each other. This is because the dog can't be black and white at the same time. Remember that we have one dog, and it can only have one color. So the system is called a contradictory system. The more information a system carries, the more useful it will be for you. For this reason, we'll introduce some terminology that you'll be using throughout the whole course. When a system is redundant or contradictory, it's called a singular system. And when a system is complete, it's called a non-singular system. In a nutshell, a non-singular system is a system that carries as many pieces of information as sentences, so it's the most informative a system can be. And a singular system is less informative than a non-singular one. Systems of sentences can carry more than two sentences. In fact, they can carry as many as we want. Here are some examples of systems with three sentences. In this new example, you have three animals and are again trying to determine their color. The first system has the sentences, The dog is black, the cat is orange, and the bird is red. The second one has the sentences, The dog is black, the dog is black, and the bird is red. The third one says, The dog is black, the dog is black, and the dog is black. And finally, the fourth one has the sentences, The dog is black, the dog is white, and the bird is red. So the first system is complete, as it carries three different pieces of information using three systems. So it's complete and non-singular. The second system is redundant and singular, as the first and second sentences say the exact same thing. Can you guess what the third system is going to be? So if you guessed redundant, well done. The third system is redundant, since all the sentences say the same thing. And finally, the fourth system is contradictory, because the dog can't be black and white at the same time. Notice that the third system is more redundant than the second system, which has two sentences that say the dog is black. Is there a measure of how redundant a system is? And the answer is yes, and it's called a rank. But you'll learn this a bit later this week. And in terms of singularity and non-singularity, the terminology is exactly like before. The first system is non-singular, as it is complete, and the other three systems are singular, as they are either redundant or contradictory. Now, systems can be a bit more complicated than the ones we previously saw. Consider the system of sentences. Sentence 1 says, Between the dog, the cat, and the bird, one of them is red. Sentence 2 says, Between the dog and the cat, one of them is orange. And sentence 3 says, The dog is black. So, problem 1 of this quiz says, Can you figure out what color is the bird? And problem 2 says, Is this system singular or non-singular? Great job! And the solution for question 1 is that the bird is red. Why? Well, if you look at the third sentence, it says that the dog is black. So now you know that the dog is black in the entire system of sentences. When you look at the second sentence, it says that between the dog and the cat, one of them is orange. Since the dog is black, then the cat must be orange. And finally, the first sentence says that among the three animals, one of them is red. Since the dog is black and the cat is orange, then we must conclude that the bird must be red. For question 2, since you figured out the color of the three animals, that means the system carries three pieces of information with three sentences. In other words, it has no redundancies and no contradictions. It carries as many pieces of information as sentences. Therefore, it is a complete system and it is non-singular.