"Therefore, I think that the failure of deeper networks to achieve better results is an optimization problem, not a model design problem, or a model capability problem. The model itself has greater potential, but the optimization method needs to change. "
“And that’s what deep residuals are about.”
"For any mapping H(x) we need to learn, I expect the network to learn F(x) in F(x) + x = H(x), rather than directly learning H(x) itself."
"This operation can be simply implemented by adding an addition, and the difference between the distance H(x) is F(x) of x itself. We call it the residual mapping of identity."
"If this identity is ideal, then we can easily set the weight to a very small value. The form of the residual solves the gradient problem that has always been troublesome. It can be seen that it makes hundreds of layers The network can consistently achieve better performance."
"The results achieved in the recognition and classification of this competition are only the most basic embodiment of the residual idea. In fact, it is a better feature extractor that can better extract the features of images and apply them to various image tasks. go."
"Not only on the test track of this competition, but also at Baidu's technical conference two days ago, everyone must have seen obvious performance improvements."
Having said this, Meng Fanqi paused for a moment, because the audience could not control themselves and began to talk about it.
Although Baidu announced that the real-time detection method was mainly contributed by Meng Fanqi, a special researcher, it remained tight-lipped about the specific algorithm details and mentioned nothing.
He just promised to wait until 6-12 months before announcing it.
Although everyone had probably guessed it, but hearing what Meng Fanqi said personally, everyone was sure of at least one thing, that is, Baidu's real-time detection algorithm, which is now leading the world by a large margin, is based on DreamNet.
"Of course, there are many more visual tasks. In addition to the generation, detection, segmentation, and recognition that I have already done, there are also pose estimation, depth estimation, super-resolution, etc. There are many different types. And enough."
"My DreamNet paper has been published and the code has been open sourced. There are still many directions for everyone to explore together."
Meng Fanqi ate the cakes from the biggest tracks, so naturally he had to leave some soup for others.
In a relatively specific direction, it is enough to have a few highly recognized masterpieces.
Meng Fanqi has so many technologies that he can’t finish them all, so there is no need to do all the subdivided fields by himself.
To put it bluntly, it means making slight changes to make the machine run, changing the data, and slightly adjusting individual structures and parameters.
It is more cost-effective to open source the code and allow more and more work to be performed based on your own technology and algorithms.
It actually took Meng Fanqi only ten minutes in total to talk about this point.
According to the original plan, he could talk for about 25-30 minutes.
It's a pity that my pockets have bulged, my courage has become stronger, and my mentality has changed. Meng Fanqi no longer has the psychological need for academic recognition and recognition that he had before meeting Robin Li.
In retrospect, before Robin Li's advance was credited to his account, Meng Fanqi had always been worried. He always felt a little uneasy and wanted to be recognized.
Some doubt how many resources they can leverage.
Now, these are all things of the past. Without the need to be recognized, Meng Fanqi's performance has become much more refined.
Ten minutes is actually not a short time in this kind of situation, especially this presentation has two parts. After Meng Fanqi finished, there was also a theoretical explanation part of Han Ci.
Therefore, no one present felt anything unusual. Only Han Ci was stunned. What's going on? Shouldn't there be ten more minutes? Why is it my turn?
“When I was doing these studies, I received a lot of help from Fu Deqing, a mathematics professor in our school. He was a collaborator on the paper, but he was not a scholar in our field, so he was not willing to participate in this conference.
Regarding why the residual idea works and what its actual significance is, we asked Professor Fu’s junior sister, Han Ci, to give us her views and explanation from the perspective of dynamic systems. "
After Meng Fanqi finished speaking, he was about to walk off the stage. After taking two steps, he turned back and added something to the microphone.
"By the way, since I signed a contract with Google, and considering my current academic situation and I am still studying for an undergraduate degree, I urgently need a university near Silicon Valley to take me in."
"I hope Stanford can be more conscious."
After saying that, the whole audience burst into laughter.
The participating teams this time are Microsoft, UC Berkeley, St. Petersburg, IBM, Tokyo University, National University of Singapore, Oxford, and Toronto.
The level of participants is also high, and most of them are master's and doctoral degree holders from world-class top universities and technology giants.
The results of these people were blown up in several streets. Crowds of people gathered here to listen to Meng Fanqi introduce his algorithm. This caused many people present who did not know the inside story to completely ignore Meng Fanqi's status as an undergraduate student.
The quality and standard of the papers he has published are sufficient for doctoral graduation.
Network structure, generation, segmentation, optimizers and normalization methods are all visible to anyone with a discerning eye. These ideas will become the basic paradigm of the new AI era.
There are even foundation-laying and digging-type digging works in two or three new directions.
No one thought that he still had such a problem to solve.
After Meng Fanqi said these words, Li Feifei, the data preparer of the event and one of the directors of the Stanford AI Laboratory, directly started on-site enrollment.
When she collected IMAGENET data a few years ago, because the algorithm's capabilities were too far behind the human level, Li Feifei had always hoped that one day the AI algorithm would surpass humans in the large-scale data she collected.
She originally thought it would take ten or twenty years, but she never imagined that it would take less than five or six years.
Especially in the past two years, the accuracy has been increased by 20%, which directly fulfilled her wish.
Now that he has gotten what he wanted, and the person who did it happens to be seeking the opportunity to study at Stanford, Li Feifei will naturally not let it go.
Moreover, in 2014, Stanford was about to start offering courses in deep learning, and the addition of Meng Fanqi was also greatly beneficial to this matter.
Li Feifei was thinking this and didn't feel that her idea was a bit strange. She wanted to recruit an undergraduate student but wanted him to help the university provide course quality.
I really don’t know whether they are recruiting undergraduates or lecturers. This seems a bit absurd.
There were many professors and scholars from Oxford, Cambridge, MIT, Harvard and other prestigious universities present. They originally heard that he wanted to study in the United States, and they all wanted to recruit him.
But after hearing Meng Fanqi say that he signed a contract with Google, and also named Stanford to be more conscious, several old professors still have some baggage.
Seeing Li Feifei and Meng Fanqi chatting and laughing, they were embarrassed to step forward and interrupt.
