"If you don't want to major in electrical, you can get to the top of your grade and then apply to school to change your major. Instead of giving up like this." Xiao Jiebao acted earnestly, but Meng Fanqi knew very well that he was just worried about giving advice. Things like quitting may affect his performance as an instructor.
"This is a contradiction in itself. I don't want to learn or cannot learn a major, but I have to get to the top in this major before I have the opportunity to change majors." Meng Fanqi can understand that this rule is to limit large-scale major changes. behavior, or to protect some unpopular subjects.
You may be able to restrict him from changing his major, but will he engage in this field of work after graduation? Or even say, are there so many jobs in this field?
In this way, many people are stuck in a major they don't want to study. They can only change industries through postgraduate entrance examinations or training courses in society, so that a certain major in the university seems to be prosperous, but in fact it is just a cover-up.
"Besides, teacher, I have not given up on myself. In the past period of time, I have been working hard to learn and research cutting-edge knowledge in new fields." Meng Fanqi pointed to his computer screen, which was his paper on "Batch Normalization" The draft mainly discusses the internal covariate shift problem during the training process of deep neural network.
"If it doesn't work, I want to take a year off from school." Meng Fanqi has also considered this issue. On the one hand, he has given up on electrical subjects, and on the other hand, he does not have the capital to transfer to the corresponding major or continue studying in other universities.
By taking a leave of absence, some trivial matters at school and college will not affect you. But considering that when suspending school, it is difficult for the school not to know about his parents. Meng Fanqi thought about it for a while, but still did not make this decision.
His parents have sacrificed a lot for him, and it is difficult to come up with evidence that the elders can understand to reassure them. Meng Fanqi does not want to affect his parents' mood and daily life.
"Don't fool me with some English literature." In fact, the instructor already believed what he said when he saw this, but he had already boasted to the college leaders this year that students would never be dismissed from school or dismissed. Things like this happen. "If you are still so stubborn, I can only ask your parents to come to school."
Meng Fanqi's eyes turned cold when he heard this. If the instructor really had him in mind, this matter could not be explained. But he knew very well in his heart that this was probably his own Xiao Jiujiu.
"Don't go to such trouble, isn't it because students' interests have changed? We who work in universities should pay attention to students' ideas and needs. Provide students with a better environment and conditions. We will trouble students' parents at every turn. What does it look like? The draft on this classmate’s computer screen is interesting to me.”
A middle-aged man with an elegant face appeared behind the instructor at some point. He was dressed a little more elegantly, wearing a pair of rimless gold-rimmed glasses, and his hair was slicked back.
"Dean Fu?" The instructor saw the face of the visitor and was startled, "Why are you here?"
Fu Deqing is the dean of the School of Mathematics and Physics of Yanjing Electric Power College. Although the School of Mathematics and Physics is not a large institution in the school, Dean Fu also holds an important position in the school party committee, so he has a high status.
When the instructor who always wanted to climb up saw Dean Fu, he immediately stopped and said with a smile, "Let Dean Fu handle everything. I won't disturb you. I'm leaving first."
Meng Fanqi stood up quickly. He could see that Dean Fu's visit was not accidental, and it was most likely that he came here specifically to see him.
But after thinking about it, I couldn't think of anything about myself that would attract the dean of another hospital to visit me.
"The internal covariate shift in deep neural network training will cause the deeper input distribution to be seriously affected. In order to adapt to the shifted data distribution, the deep neural network has to use a conservative way to optimize parameters, thus greatly affecting It affects the training speed and training difficulty." Dean Fu did not explain this, but just looked at the content on Meng Fanqi's screen.
"The deep neural network you are talking about here should refer to the hundreds or thousands of layers of neural networks you proposed yourself, not the current deep network in the industry, right?" Dean Fu turned his head and asked with a smile.
"Dean Fu, how did you..." Meng Fanqi was stunned for a moment, and then suddenly realized that only that girl had seen the draft of the residual network that she had prepared in advance. "Did that girl tell you?"
"Hahaha, she is my junior sister." Dean Fu said with a smile, "He came here some time ago and stopped by to visit me. He mentioned that there is a student in our school who is studying a neural network with hundreds of layers. , researched it decently."
"To be honest, this is not my field. If my previous teacher hadn't asked me to pay attention to this area, I would really know nothing about it. I have been looking up information these two days, and I said there are hundreds of layers of nerves in this. Network, AlexNet, which won the championship at the end of last year, was only eight layers deep."
"I just had some ideas." Meng Fanqi quickly explained.
"Don't be nervous. I'm just a little curious and want to see if there is a genius in this field in our school. If it is true, I will definitely support it."
Meng Fanqi quickly asked Dean Fu to sit down and gave him a brief introduction to the background, principles and current problems.
Dean Fu had a background in pure mathematics, and he had learned a little bit about this field under the instructions of his teacher. Therefore, Meng Fanqi didn't say much, and he had already grasped the general outline.
"You think the problem that is difficult to train deep networks is essentially a gradient problem, so you want to use the form of residuals." Dean Fu muttered to himself, making a fist with his left hand and supporting his chin.
"If you don't delve into the transformation of the neural network itself, but just understand its transformation as an unknown high-dimensional function, the multiple accumulations of f(x) will be more unstable than the residual form of x+f(x) Too many." Meng Fanqi's later generations mainly focused on understanding the process of the algorithm and understanding the advantages and disadvantages of several comparison methods.
As for its specific internal principles and mathematical derivation, to be fair, he did not care enough. Now is a great opportunity to make up for these shortcomings.
"If we discuss multiple nestings of a function, f(f(f(f(..f(x)..)))), whether it is in the form of residuals or not, it is quite difficult to analyze. From Mathematically, intuitively, two nestings of the same degree, with or without residuals, should not change the actual expression ability of the function. In other words, the ability of the model you build to fit the function is equivalent to the original one. of."
"If you can really observe a significant improvement, it means that this form of residuals just makes the model you are talking about easier to optimize. In other words, from my perspective, the solution of the function becomes more accurate , from this perspective, the residual is a concise Euler forward form."
Fu Deqing said, picked up a pen and paper and started some deductions. Meng Fanqi was unfamiliar with mathematical deductions, but he was very familiar with the characteristics of networks constructed in this way, and the more he listened, the more surprised he became. Dean Fu's mathematical analysis turned out to be quite close to many later experimental results.
It's a pity that it's a little difficult for me to see him making deductions in person.
At this moment, just as confused as him were several senior seniors nearby who were pricking up their ears and eating melons.
