Sign in to continue learning
Unsupervised Learning, Recommenders, Reinforcement Learning
27 hours 16 mins
When you have a few positive examples with y equals 1 and a large number of negative examples, say y equals 0, when should you use anomaly detection and when should you use supervised learning? The decision is actually quite subtle in some applications, so let me share with you some thoughts and some suggestions for how to pick between these two types of algorithms. An anomaly detection algorithm will typically be the more appropriate choice when you have a very small number of positive examples, 0 to 20 positive examples is not uncommon, and a relatively large number of negative examples with which to try to build a model for p of x, where you recall that the parameters for p of x are learned only from the negative examples, and this much smaller set of positive examples is only used in your cross-validation set and test sets for parameter tuning and for evaluation. In contrast, if you have a larger number of positive and negative examples, then supervised learning might be more applicable. Now, even if you have only 20 positive training examples, it might be okay to apply a supervised learning algorithm, but it turns out that the way anomaly detection looks at the dataset versus the way supervised learning looks at the dataset are quite different. Here's the main difference, which is that if you think there are many different types of anomalies or many different types of positive examples, then anomaly detection might be more appropriate. When there are many different ways for an aircraft engine to go wrong, and if tomorrow there may be a brand new way for an aircraft engine to have something wrong with it, then your 20, say, positive examples may not cover all of the ways that an aircraft engine could go wrong. That makes it hard for any algorithm to learn from the small set of positive examples what the anomalies, what the positive examples look like, and future anomalies may look nothing like any of the anomalous examples we've seen so far. If you believe this to be true for your problem, then I would gravitate toward using an anomaly detection algorithm, because what anomaly detection does is it looks at the normal examples, that is, the y equals zero negative examples, and just tries to model what they look like. And anything that deviates a lot from normal, it flags as an anomaly, including if there's a brand new way for an aircraft engine to fail that had never been seen before in your dataset. In contrast, supervised learning has a different way of looking at the problem. When you apply supervised learning, ideally, you would hope to have enough positive examples for the algorithm to get a sense of what the positive examples are like. And with supervised learning, we tend to assume that future positive examples are likely to be similar to the ones in the training set. Let me illustrate this with one example. If you are using a system to find, say, financial fraud, there are many different ways, unfortunately, that some individuals are trying to commit financial fraud. And unfortunately, there are new types of financial fraud attempts every few months or every year. And what that means is that because they keep on popping up completely new and unique forms of financial fraud, anomaly detection is often used to just look for anything that's different than transactions we've seen in the past. In contrast, if you look at the problem of email spam detection, well, there are many different types of spam email, but even over many years, spam emails keep on trying to sell similar things or get you to go to similar websites and so on. Spam email that you will get in the next few days is much more likely to be similar to spam emails that you have seen in the past. So that's why supervised learning works well for spam because it's trying to detect more of the types of spam emails that you have probably seen in the past in your training set. Whereas, if you're trying to detect brand new types of fraud that have never been seen before, then anomaly detection may be more applicable. So let's go through a few more examples. We have already seen fraud detection being one use case of anomaly detection, although supervised learning is used to define previously observed forms of fraud, and we've seen email spam classification typically being addressed using supervised learning. You've also seen the example of manufacturing, where you may want to find new previously unseen defects, such as if there are brand new ways for an aircraft engine to fail in the future that you still want to detect, even if you don't have any positive example like that in your training set. It turns out that in manufacturing, supervised learning is also used to find defects, more for finding known and previously seen defects. For example, if you are a smartphone maker, you're making cell phones, and you know that occasionally your machine for making the case of the smartphone will accidentally scratch the cover. So scratches are a common defect on smartphones, and so you can get enough training examples of scratched smartphones corresponding to a label y equals one, and just train the system to decide if a new smartphone that you just manufactured has any scratches in it. And the difference is, if you just see scratched smartphones over and over, and you want to check if your phones are scratched, then supervised learning works well. Whereas if you suspect that there are going to be brand new ways for something to go wrong in the future, then anomaly detection will work well. Some other examples, you've heard me talk about monitoring machines in the data center, especially the machines in the hack. It can behave differently in a brand new way, unlike any previous way it has behaved. So that would feel more like an anomaly detection application. In fact, one theme is that many security-related applications, because hackers are often finding brand new ways to hack into systems, many security-related applications will use anomaly detection. Whereas returning to supervised learning, if you want to learn to predict the weather, well, there's only a handful types of weather that you typically see. Is it sunny, rainy, is it going to snow? And so because you see the same upper labels over and over, weather prediction would tend to be a supervised learning task. Or if you want to use the symptoms of the patient to see if the patient has a specific disease that you've seen before, then that would also tend to be a supervised learning application. So I hope that gives you a framework for deciding when you have a small set of positive examples, as well as maybe a large set of negative examples, whether to use anomaly detection or supervised learning. Anomaly detection tries to find brand new positive examples that may be unlike anything you've seen before. Whereas supervised learning looks at your positive examples and tries to decide if a future example is similar to the positive examples that you've already seen. Now it turns out that when building an anomaly detection algorithm, the choice of features is very important. And when building anomaly detection systems, I often spend a bit of time trying to tune the features I use for the system. In the next video, let me share some practical tips on how to tune the features you feed to your anomaly detection algorithm.