"Mom, I won't go back this year. If everything goes well, I'll go back to visit you early next year."
"I'm not playing around outside. I'm not dropping out of school. Don't be nervous, I know what I'm doing."
"There is no need to give me any money. I am not starting a business. I just need to concentrate on solving some problems during this period. It will be the end of this year. Don't worry, it will be the end of this year. If it doesn't succeed, I will listen to you whatever you say."
"Okay, okay, remember to ask dad to go out and exercise more. Take care of your health!"
"Go back, go back. I must go back during the Chinese New Year."
After hanging up the phone, Meng Fanqi leaned against the wall and let out a long sigh.
It was really hard to explain these things to his parents. If it weren't for the worry that Dean Fu would help, he would not have planned to take a leave of absence from school.
Now that parents know the current situation, they must be worried. In the last few phone calls, no matter what I said, my parents couldn't let go.
Meng Fanqi could only rely on the deadline at the end of the year to appease them, promising that as long as he could not achieve outstanding results this year, he would listen to them in everything.
Looking at Don Juan who was gloating about his misfortune and adjusting the ginseng in the room, Meng Fanqi said angrily, "Why are you laughing?"
Don Juan instantly made a serious expression, but no matter how he looked at it, he still had a sense of yin and yang, which made people very uncomfortable.
--------------------------
Later that day, Meng Fanqi received Alex's reply. Looking at the time, Alex should have replied to the email as soon as he got up. It can be said that he was quite diligent.
This time point is close to the start of this year's event, but Alex himself has no plans to continue to form a team to participate in this event.
It's just that although I'm not in the world, the world is full of legends about me.
The 2013 Microsoft Research team, the National University of Singapore team, and the Decaf, UC Berkeley and other teams led by Jia Yangqing were all deeply influenced by Alex's thoughts last year.
According to the method introduced by Alex, Meng Fanqi quickly completed the debugging, successfully migrated batch normalization, Adam optimizer and other operations from his own experimental environment to the cuda-convnet framework, and made some adjustments and adaptations.
It didn't take long before it was already running on CIFAR-10.
Next is the core competitiveness of this competition, the implementation of residual network on this framework.
"This year's IMAGENET Challenge actually doesn't have many new ideas. Everyone basically focuses on replicating Alix's algorithm."
Of course, Meng Fanqi is not prepared to re-implement Alex's competition algorithm last year based on Alex's framework like other teams. Many early algorithms have a lot of redundancy in terms of parameters and calculations.
"Google's GoogleNet and Oxford's VGGNet were not supposed to be developed until next year, but after I participate in the competition this year, these two important networks in the history of AI will probably be aborted, right?"
Not to mention now, as of the end of 2014, redundant designs still exist in the algorithms of Google and Oxford University.
It wasn't until 2015 that that man won ImageNet-2015 with the residual network ResNet, creating the godhead of AI models in the deep learning era.
Residual thinking is Meng Fanqi's best choice at this moment. On the one hand, it will become a milestone in the era of deep learning, because its core idea is how to make deep networks 'deeper'.
On the other hand, its implementation and structure are simple, abandoning a large number of designs that seem reasonable and valuable to humans, but are actually of little use.
Repeating the simple and easy-to-use structure repeatedly also greatly reduced the amount of development engineering required by Meng Fanqi.
AlexNet has an eight-layer structure, each with some independent designs. At this time, the 8-layer network is already a revolutionary deep network.
Deeper networks are particularly difficult to train, so it was not until 2014 that Google and Oxford advanced the depth to 22 and 19 layers respectively.
The idea of ResNet completely and fundamentally solves the problem that the network cannot be trained smoothly as the network becomes deeper. It makes it possible to train 50, 100, 150, or even 1000-layer networks.
"From 8 last year, to 100+ floors this year, you can even train at 1,000 floors. From the perspective of 2013, this strong dose should be enough."
Not only has it achieved groundbreaking breakthroughs in depth, ResNet's performance is also amazing. It is the first structure with a Top-5 error rate lower than human capabilities, with a single model reaching about 4.6%.
If several ResNets trained in different ways are combined and their outputs are averaged to predict, the TOP-5 error rate is even as low as 3.7%.
"Actually, it's a weird thing to be too accurate on IMAGENET." Considering later research, a large proportion of the labels in this data set are actually wrong, and the closer the accuracy is to 100%, the more absurd it is. a feeling of.
The Res of ResNet refers to residual, which is more seriously called residual. To put it more clearly, it is a kind of short circuit or jump link.
To put it more simply, assuming that the original operation is f(x), ResNet changes the calculation process from f(x) to f(x)+x.
This was also one of the first discussions Meng Fanqi had with Dean Fu.
This kind of branch link without any parameters directly adds the content before the operation to the content after the operation. This is the secret why the deep network can suddenly converge at a depth of thousands of layers.
Almost all AI articles after this are difficult to get around this idea. This article alone by Kai Ming received 150,000 to 60,000 citations in six or seven years.
I have to say that this is an astronomical figure. You must know that even if one of the three world-class journals of Nature Cell Science is a frequent visitor and a famous double scientific academician from China and the United States, the total number of citations of all articles so far is over 50,000, which is considered too much.
Of course, it is biased to evaluate the quality of an article and the author based solely on the number of times a paper has been cited.
In many basic subjects, there are only a few people who read and write about them. Naturally, the citations cannot be uploaded.
But for AI, a subject with a strong practicality and experimental nature, if we compare it internally, the gold content of the references is still quite high.
Everyone will only vote with code, and valuable things will continue to be iterated. Technologies that help improve the performance of your own model on tasks will continue to be cited.
There has always been a saying in academia that those who engage in theory and those who engage in experimentation look down upon each other. This is not something that came out of nowhere.
After Kaiming established his historical position with several masterpieces, there are always people who like to say, "The ideas of ResNet have been mentioned long ago", "Things from decades ago are nothing new", "This article That’s just high traffic, just like traffic stars.”
The sour smell almost overflows the screen.
It’s not that Meng Fanqi hasn’t read some theoretical articles, but he personally feels that in the new era of AI, everyone can tell you that writing code is the key. You said something existed decades ago, why not write a ResNet to break through human levels?
Is the core code of ResNet difficult to write? It's not difficult.
Based on Alix's framework, he could basically reproduce all the structures in the ResNet paper in just a few hours.
Weakness and ignorance are not obstacles to survival, arrogance is.
