Sign in to continue learning
Generative AI for Everyoneㅤ
Beginner
5 hours
Topics
Fine-Tuning
GenAI Applications
Generative Models
Prompt Engineering
In a short time, access to generative AI has spread around the world and given many people the ability to generate high quality essays, pictures, and audio. With these amazing capabilities have also come many concerns about AI. I think even before the rise of generative AI, we've been living in a time of many anxieties. Anxieties about the environment, about the legitimacy, incompetence of authority, about society's ability to treat people fairly, even about what sort of future awaits us all. AI as a very powerful technology has inherited a large share of this anxiety. In this video, let's take a look at some of these anxieties and concern that relate specifically to AI. One widely held concern about AI is whether it might amplify humanity's worst impulses. LLMs are trained on text from the Internet, which reflects some of humanity's best qualities, but also some of its worst, including some of our prejudices, hatreds, and misconceptions. LLMs learn some of these negative qualities, too. So will it amplify our worst impulses? In the first week, we had seen an example of an LLM exhibiting a gender bias with regard to whether a surgeon or a nurse is more likely to be male or female. To take another maybe slightly simpler example, if you asked an LLM after its initial training to fill in the blank and the blank was a CEO, many models would be prone to choose the word man. And, of course, this is a social bias that distorts the fact that people of all genders can successfully lead companies. Text on the Internet represents our present and our past. And so perhaps it's no surprise that an LLM learning from this data reflects some of these biases from our past and our present as well. But perhaps we want LLMs to represent a hopeful future that is fairer, less biased, and more just, rather than just data from our past. Fortunately, LLMs are becoming less biased through fine-tuning, which we discussed in week two. As well as more advanced techniques, such as reinforcement learning from human feedback or RLHF. In the second week, there was an optional video on RLHF. Whether or not you watched that, I'd like to briefly describe how RLHF is helping to make LLMs less biased. RLHF is a technique that trains an LLM to generate responses that are more aligned with human preferences. The first step of RLHF is to train an answer quality model called a reward model that automatically scores answers. So in this step of RLHF, we would prompt the LLM with many queries like this, the blank was a CEO, and collect different responses from the LLM. Then we would ask humans to score these answers. So on a scale of one to five, we give a high score to highly desirable answers like man or woman, and a low score to nonsensical answers like airplane. And any answer that contains a gender bias or racial bias or contains a gender or racial slur will receive a very low score. Using the prompt, the responses, and the scores assigned by humans as data, we would then use supervised learning to train a reward model that can input a response and score it. We do this because asking humans to score responses is expensive. But once a supervised learning algorithm has learned to automatically score responses, we can score a lot of responses automatically and inexpensively. Finally, now that the LLM has a learned reward model to score as many responses as it wants, we can have the LLM generate a lot of responses to many different prompts. And have it further train itself to generate more responses that get high scores and that, therefore, reflect answers that humans perceive as more desirable. RLHF has been shown to make LLMs much less likely to exhibit bias according to gender, race, religion, and other human characteristics. It makes LLMs less prone to hand out harmful information, and also makes it more respectful and hopeful to people. Already today, the output of LLMs are much safer and less biased than, say, the average piece of text on the Internet. But technology like this is continuing to improve, and so the degree of an LLM amplifying humanity's worst qualities is continuing to decrease as they are becoming better aligned to the future. I think we all hope LLMs will reflect of a fairer, less biased, and more just world. A second major concern is who among us will be able to make a living when AI can do our jobs faster and cheaper than any human can? Will AI put many of us out of a job? To understand whether this is likely to happen, let's look at radiology. In 2016, many years ago, Geoff Hinton, who's a pioneer of deep learning and a friend of mine, said that AI was becoming so good at analyzing X-ray images that in five years, it could take radiologists' jobs. He made this remarkable statement that if you work as a radiologist, you're like a coyote that's already over the edge of the cliff, but hasn't yet looked down. So it doesn't realize there's no ground underneath them. People should stop training radiologists now. It's just completely obvious that within five years, deep learning is going to do better than radiologists. But we're now well past five years since this statement, and AI is far from replacing radiologists. Not a single one of my radiologist friends has lost their job to AI. Why is that? Two reasons. First, interpreting X-rays turns out to be harder than it looked back then, though we are making rapid progress. But second and more important, it turns out that radiologists do a lot more than just interpret X-ray images. According to O*NET, radiologists do about 30 different tasks, one of which is interpreting X-rays and other medical images, but they do many other tasks. And it has been difficult so far for AI to do all of these tasks at human level. To list out some of the other tasks that radiologists do, in addition to interpreting X-rays, they also operate imaging hardware, communicate exam results with patients or other stakeholders. Respond to complications during procedure, such as if a patient has a panic attack during the imaging procedure. They document procedures and outcomes, and many other tasks. And I think that AI does have a high potential of augmenting or assisting the interpretation of X-rays. And technically, this has largely been done with supervised learning rather than generative AI. But for AI to completely automate all of these tasks is still far away. So that's why I think that Curtis Langlotz, who is a professor of radiology at Stanford University and a friend and colleague, says it well. He said that AI won't replace radiologists, but radiologists that use AI will replace radiologists that don't. And I think we will see this effect in many other professions. Mind you, I don't mean to minimize the challenge of helping many people adopt AI or the suffering of the much smaller number of people whose jobs will disappear. Or our responsibility to make sure people affected have a safety net and an opportunity to learn new skills. But every wave of technology, from the steam engine to a chassis to the computer, has created far more jobs than it destroyed. As I mentioned earlier this week, in most waves of innovation, businesses wound up focusing on growth, which has unlimited potential rather than cost savings. So AI will bring a huge amount of growth and create many, many new jobs in the process. And this brings us to what might be the biggest anxiety, will AI kill us all? We know that AI can run amok. Self-driving cars have crashed, leading to a tragic loss of life. In 2010, an automated trading algorithm caused the stock market crash. And in the justice system, AI has led to unfair sentencing decisions. So we know that poorly designed software can have a dramatic impact. But can it lead to the extinction of humanity? I don't see how. I know there are different views on this, but recently, I sought out some people concerned by this question, and I spoke of some of the smartest people in AI that I know. Some were concerned about a bad actor using AI to destroy humanity, say, by creating a bioweapon. Others were worried about AI inadvertently driving humanity to extinction. Similar to how humans have driven many other species to extinction through simple lack of awareness that our actions could lead to that outcome. I tried to assess how realistic these arguments are, but I found that they were not concrete and not specific about how AI could lead to human extinction. Most of the arguments boil down to it could happen. And some will add that this is a new type of technology, so things could be different this time. But that statement is true for every new type of technology that's been invented by humanity. And proving that AI couldn't lead to human extinction is akin to proving a negative. I can't prove that AI superintelligence won't wipe out humanity, but it's just that nobody seems to know exactly how it could. But I do know this, humanity has ample experience controlling many things far more powerful than any single person, such as corporations and nation states. And also that there are many things we can't fully control that are nonetheless valuable and safe. For example, take airplanes, which today, we still can't fully control because winds and turbulence will buffet airplanes around, or the pilot flying the plane may make a mistake. In the early days of aviation, airplanes killed many people. But we learned from those experiences and built safer airplanes, and also devised better rules by which to operate them. And today, many people can step into an airplane without fearing for their lives. Similarly for AI, we are learning better to control it, and it is becoming safer every day. Finally, if we look at the real risks to humanity, such as climate change leading to massive depopulation of parts of the planet, or hopefully not the next pandemic. Or even much lower chance, but another asteroid striking the planet and wiping us out like the dinosaurs. I think that AI will be a key part of our response to such challenges. So I know that there are different views on this right now. But my view is that if we want humanity to survive and thrive for the next thousand years, AI increases the odds of us successfully getting there. Computers are already smarter in some narrow dimensions than any human. But AI continues to improve so fast that many people find it hard to predict exactly what it will be like in a few years. I think the root cause of some of these concerns, including extinction risks, is that many people are unsure when AI will reach Artificial General Intelligence, or AGI. Meaning, AI that could do any intellectual task that human can. Let's take a deeper look at AGI in the next video.
