Sign in to continue learning
Deep Learning Specialization
Intermediate
Topics
Computer Vision
Deep Learning
NLP
Supervised Learning
Transformers
Welcome, Yanting. I'm really glad you could join us today. Sure. So, you know, today you're the head of Baidu Research, and when the Chinese government, the government of China, was looking for someone to start up and build a national deep learning research lab, they tapped you to help start this thing. So, you know, arguably, I think maybe you're the number one deep learning person in the entire country of China. So I'd love to ask you a lot of questions about your work, but before that, I want to hear about your personal story. So how did you end up, you know, getting to do this work that you do? Yeah, so actually before my PhD program, my major was in kind of optics, so it's more like in physics. I think I had a fairly good kind of background, like a very good background on math. Then after I came to the U.S., then I was thinking, like, what kind of major I'm going to take for my PhD program. And I was thinking, like, whether I should go for optics or go for something else, because back to, like, early 2000, I think nanotechnology was very, very hot. But I was thinking, like, probably I should look at something, like, even more exciting. And there was a good chance that I was taking some classes at UPenn, and I got to know Dan Lee, so later he became my PhD advisor. And I was thinking, like, machine learning was a great thing to do. And I got really excited, and I kind of changed my major. So I did my PhD at UPenn, like, majoring in, like, machine learning, yeah. And so that kind of, actually, I was there for five years, and that was kind of really exciting. I learned lots of things from scratch, lots of algorithms, even, like, PCAs. I didn't know those before, yeah. It was kind of, I feel like I was learning new things every day. So it was a very, very exciting experience for me. It was one of those things with a lot of stars, although, you know, did a lot of work, and it was underappreciated for its time. Right, right, that's right, yeah. So I think NG was an exciting place, and I was there at the beginning as a researcher. Again, I also, like, feel like, wow, I learned lots of things. And actually, later at NG, I kind of started working on computer vision. I actually started working on computer vision very late, I mean, relatively late, yeah. And the first thing I did over there was I participated in the ImageNet Challenge. That was the first year of ImageNet Challenge. I was kind of managing a team to work on a project. It was lucky. We were quite lucky. We were quite strong, and at the end, we actually got the number one place, overwhelming number one place in the context. So you were the first ever in the world ImageNet competition winner. Right, right, right, yeah. And I was the person that did the presentation at the workshop. Yeah, so that was a really nice experience for me. And that actually got me into this very large-scale kind of computer vision tasks. Yeah, and so I have been working on this very large-scale problem since then. And so when this cat came out, and also later on, like, this AlexNet came out, it really blew my mind. Actually, I thought, like, wow, deep learning is so powerful. And I think since then, it was lots of effort, I think, to work on those. So as a head of China's national lab, national research lab on deep learning, there must be a lot of exciting activities going on there. So for the global audience watching this, what should they know about what's happening with this national lab? The mission of this national engineering lab is to build a very large deep learning platform, and probably hope to be the biggest one, at least the biggest one in China. And on this platform, we would offer people, like, a deep learning framework, like PEDDLE, PEDDLE. And we offer people a very large-scale computing resource. And we also offer people very large, like, very big kind of data. And if people are able to kind of develop or research or develop a kind of good technology on this platform, we also offer them, like, big applications. So, for example, the technology can be plugged into some big applications in Baidu, so that the technology could get iterated and improved. So we believe that combining those resources all together, I think this is going to be a very powerful platform. Let me also give you kind of one example on this. For example, like, right now, if we publish a paper, say, a researcher publishes a paper, if someone wants to reproduce it, then they need to kind of, the best thing to do is probably, that person will provide the code somewhere. And you could download that code to your computer. And you also try to find the data set somewhere. And you probably also need to get the good calculation of your kind of computing resource to run it smoothly, right? So this easily kind of take you some efforts. At the least, a deeper national lab, things will become much easier. So if someone is using this platform to write the paper, to do the work and write the paper, and they will have the code on this platform, and the computing structure is already set up for this code, and the data is there, too. So basically, you just need a command line to reproduce the result. So this is a huge, I think it's a bigger, bigger kind of relief for lots of reproducibility issues in computer science. So easily, you just kind of, just a few seconds, you start learning something that you see in the paper. So this is extremely powerful. So this is just one example that we are working on to make sure that we are providing a very powerful platform to the community and to the industry. That's amazing. That really speeds up deep learning research. Right. Yeah. Can you give a sense of how much resources the Chinese government is putting to back you for this deep learning national lab? So I think for this national engineering lab, I think the government is definitely going to invest some funding into here to build up the infrastructure. But I think more important would be this is going to be a flagship in China that is going to lead lots of kind of deep learning efforts, including like a national project and lots of kind of policies and things. So this is actually extremely powerful. And I think for Baidu, we are really honored to get this lab. You're somewhat at the heart of deep learning in China. So there's a lot of activity in China that the global audience isn't aware of yet and hasn't seen yet. So what should people outside China know about deep learning in China? Yeah, I think in China, especially in the past few years, I think deep learning powered product and service is really booming. Coming ranging from search engines to face recognition, to surveillance, to e-commerce, lots of places. I think they are investing big effort in deep learning and also really making use of technology to make their business much more powerful. And this actually is very important for developing AI technology in general. I think for myself, I know also lots of people share this. We believe that actually it's really important to have this, what we often call the positive loop. For example, when we start out to think of building some technology, we will have some initial data and we try to develop some initial algorithm. We will launch some initial product, product or service. Then after that, we will get the data from users. And after we get more data, we will develop better algorithms. Because we see more data, we know what will be the better algorithm. So we have more data and better algorithm, we are able to have better technology for the product and service. And then definitely we hope to be able to attract more users because of the product, the technology is better. And we will get more data. So this is a very good positive loop. And it's very special actually for AI-related technologies. For traditional technology like laser, the laser that I was working on before. So it's more like the growth of the technology is going to be more linear. But for this AI technology, because of this positive loop, you can imagine that definitely at a certain point you will come with a very fast growth of the technology. And this is actually super important. When we design the research, when we design our R&D, we work on the direction that we are able to get to this quick kind of improving period. If our whole business is not able to find this positive loop, or if we are not able to find this strong positive loop, then this direction probably should not work out. Because someone else who has a strong business to find this strong loop, then they will get to this phase much more quicker than you are. So this is actually a very important logic for us when we're looking at it. Like I said, hey, in this company, what direction we should work on and what direction we should not work on. This is definitely a very important factor we should look at. Today, both in China and in the US and globally, there are a lot of people wanting to enter deep learning and wanting to enter AI. What advice would you have for someone that wants to get into this field? So nowadays, definitely, I think people who start with an open source framework, I think that's extremely powerful to many starters. I think when I was starting my deep learning study, there was not much like an open source resource. I think nowadays, in AI, especially in deep learning, there is a very good community out there. And there are multiple very good deep learning frameworks. It starts with TensorFlow, CAFE, now it's called CAFE2. And in China, we have a very good PaddlePaddle. And even for most of those, online, there are lots of courses to teach you how to use those. And also, nowadays, there are many publicly available benchmarks. And people could see, hey, very skillful, very experienced people, how well they could do on those benchmarks. So basically, to get familiar with deep learning, I think those are very good starting points. How did you gain that understanding? So actually, I do it in the opposite way. I learned PCA, I learned LDA, all those before I learned deep learning, actually. So basically, that also is a good way, I feel. We kind of lay down lots of foundations. We learn graphical models. These are all kind of important. Right now, deep learning is very dominant. But knowing those actually gives you very good intuition about how deep learning works. And one day, probably, there's a connection of deep learning to those, like a framework approach. I think there's already lots of connections. And those actually make deep learning richer. I mean, there are richer ways of doing deep learning. So I feel like it's good to start with those open-source platforms. Those are definitely extremely powerful resources. But meanwhile, I would also suggest that you also learn those basic things about machine learning. So thank you. That was fascinating. Even though I've known you for a long time, there are a lot of details you're thinking that I didn't realize until now. So thank you very much. Thank you so much for having me.
