Generally speaking, the more mature competition lists are divided into two types, one is the public list and the other is the final private list.
The corresponding data are also different and divided into verification set and test set.
The real answers to these two parts of the data will not be published, but after the contestants submit their results, the public list will only publish the results of the verification set for everyone's reference, but will not publish the results and content of the final test set.
This is because the real-time ranking is only to help everyone understand the approximate level of their own algorithm and how far it is from the strongest algorithm. Although the answer is not provided directly, if the contestants adjust the settings and submit the results repeatedly, the content and distribution of this part of the data can still be analyzed to a certain extent.
Therefore, this part of the data is for reference only, and only the test set ultimately determines the ranking. Therefore, the public ranking list of the event can only reflect the problem to a certain extent, and cannot express the final ranking 100%.
"Although some competition lists are not used for the final ranking at all, people are easily attracted by the magic of such lists." Meng Fanqi remembered that later generations and Tang Juan participated in two small competitions, and he didn't know why he was busy all day long. I want to stare hard at that list.
If your submission score is one grade higher or one grade lower, it will cause huge fluctuations in your mood.
"It's like this in all industries, including the novel industry, the film and television drama industry, and the celebrity industry. It creates anxiety." Don Juan disagreed, "If there is no anxiety, just make a list to create anxiety. Everyone wants to be the best. As soon as this list came out, it was like a handful of bait was thrown into a calm pool. The fish that were originally turning their bellies and not moving became all active."
"Sports circles also like to talk about who is the GOAT (the strongest in history), James vs. Kobe, Messi vs. Ronaldo." Don Juan continued to complain. The sports circle is now suffering from this kind of trend, and it is about to change. It has become a fandom, "The statistical data are becoming more and more outrageous. In the past, we only counted one goal, but now we also count the goal from which part. I also saw someone saying that Ronaldo's younger brother scored a goal a few days ago." Got a ball.”
Meng Fanqi was checking the submission result information and was stunned when he heard it at first, "Does Cristiano Ronaldo have a younger brother?"
After thinking about it carefully, I realized that it was the second brother who had been doing it for a long time.
In fact, the submission website was announced on November 11th. The submission window for this year was very short, and it was not like many subsequent competitions that separated validation sets and opened public list submissions during the competition.
Submission of results will close on November 13th.
Unconsciously, another forty or fifty days passed, and Meng Fanqi polished these papers several times.
Not only that, when he later discovered that the experiments in the paper had been completed, he connected the detection algorithm to the classification model that had been trained for a long time, and ran the detection event data again.
The detection task is an advanced step of the classification task. After your program identifies the category of the picture, a further step is to use a rectangular frame to encircle the position of the object in the picture. That is the frame on the face that everyone is familiar with later.
The next step is segmentation. Instead of using large, regular graphics like rectangular frames, the detailed outline of an object is expressed on the picture at the pixel level, which is an operation similar to automatic cutout.
Of course, whether it is detection or segmentation, it is necessary to manually label the original answers of the training set.
The detection track data set of IMAGENET-2013 is not too large, with a total of nearly 400,000 images, divided into 200 categories. This advanced type of data is much harder to label, so the amount of data and classification are not the same.
However, compared with the 5,717 photos in 2012, it is already a huge leap of a hundred times in just one year.
"I didn't expect that there would be so much time." Meng Fanqi remembered that most detections at this time were still based on traditional HOG and LBP methods. The highest mAP on this 2013 data set was only about 0.225.
Since I had time to complete the experiments in my thesis, I naturally had to take the time to reduce dimensionality and attack these antique methods.
Each participating team has three chances to submit on each task, and Meng Fanqi only needs one chance.
Teams participating in the competition often train several versions of the model, then integrate some permutations and combinations, and submit them multiple times to ensure that their results will not be affected by some unstable factors.
This is also a way to pursue higher performance, because no one can guarantee which of their results will have the best performance based on location data.
Sometimes the difference between the first and second place is just a millimeter, maybe only two or three decimal places.
It's just that Meng Fanqi has no need to do this.
There was no time to do anything else in the remaining time. Meng Fanqi originally wanted to submit the results early on the 11th. It would be better to do less than to do more.
But Don Juan stopped him, saying that a hero always has to arrive at the last step, which makes it particularly dramatic.
"This submission is not displayed in real time, but the results are announced uniformly on the 14th." Meng Fanqi pointed out such an embarrassing problem.
"Well..." Tang Juan had to force himself to explain, "Although others can't see it, the organizers can. At the last moment, give them a little shock to China!"
-------------------
Across the ocean, Stanford University's AI laboratory SAIL was founded in 1963 during the first wave of neural networks. Witnessed two booms and two recessions in AI.
Today, it is directed by Li Feifei, a young Chinese scientist and organizer of IMAGENET.
Li Feifei was still at Princeton when she started the IMAGENET project in 2009. She later came to Stanford, was promoted to a tenured professor, and started leading Stanford's AI laboratory this year.
It is not an easy task to take over such a historical laboratory. In addition, this year's IMAGENET competition has just ended, and Li Feifei is very busy at the moment.
She took a look at the results for the new year yesterday, and it was as expected.
I haven’t seen any particularly groundbreaking papers this year. We are basically still learning about AlexNet and exploring new tracks.
Deep neural networks stood out last year, outperforming the rest, but who can be 100% sure that this is the right path?
Even the best-performing model still has a Top-5 error rate of more than 11 points, and generally speaking, this result may be predicted by the integration of multiple networks. It is just easy to use by ranking. This method does not have practical application value. Li Feifei does not want IMAGENET, which she built by herself, to become a brush paradise.
There is a long way to go.
At this moment, his phone suddenly rang. Li Feifei picked up the phone and took a look. It was Deng Jia.
"Holy shit, teacher, please look at the verification results of the competition."
Deng Jia's voice sounded very excited, and he said "shit" as soon as he came up.
"Result? What result." Li Feifei didn't know what happened yet. She had already seen the list yesterday. Everyone's level was about the same. What could be counted today?
It was not convenient to re-link to the server at the moment, so Li Feifei said, "Just take a screenshot and send it to me."
"beep..."
He immediately hung up over there. Li Feifei frowned slightly, wondering what happened to this kid today. He was not usually so irritable.
Soon, two pictures were sent over.
Li Feifei opened them one by one. In an instant, his pupils dilated slightly, and his breathing suddenly became rapid involuntarily.
I saw that the top row of both lists was the same team.
Team name: Dream.
The submission descriptions differed by only one letter: "A single DreamNet." and "A single DreamDet."
Among a group of submissions that integrate multiple models, the word single stands out.
