AI technology is a relatively rare subject where the application is far greater than the theory, but it is not without theory.
All the technologies Meng Fanqi has produced so far still contain many theoretical arguments and deductions in his papers.
However, most of this content comes from discussions with Dean Fu. It is just the icing on the cake and is not Meng Fanqi's original intention.
The reason why these mathematical deductions are added is mainly because the early AI community still attaches great importance to arguments in this area.
This is why those old scholars’ eyes lit up and their mouths were filled with admiration when they heard the content of Han Ci.
Although I admired Meng Fanqi's world-class experimental breakthroughs, I didn't feel this kind of sincere excitement.
For those who are obsessed with theory, understanding the theoretical reasons for this phenomenon is far more important and more attractive than making technological applications that affect the world.
It is this kind of curiosity and exploration of truth that has created breakthroughs in human civilization and technology one after another.
It's just a pity that on the road of AI, the theoretical direction is destined to be extremely bumpy.
At least until around 2023-2024, there will still be no decent breakthrough.
In Meng Fanqi's future papers, there will be fewer and fewer theoretical parts, and he will pay more attention to the difficulties and content of industrial applications.
"Who is she? Your classmate?" After listening to Han Ci's explanation of Meng Fanqi's residual thoughts, Hinton felt that his ideas suddenly opened up a lot.
If equivalence is constructed from the perspective of dynamical systems, many concepts from mathematics and physics can be introduced, and things will be promising.
"She is from Yenching University and is now a graduate student." Meng Fanqi meant that Han Ci already had a mentor.
"She should be engaged in applied mathematics." Li Feifei was not as strict about etiquette as Hinton.
In her opinion, as long as the foundation is well dug, no student can find it. "Who is her mentor?"
"Academician E Weinan." Meng Fanqi suddenly thought that Li Feifei had an undergraduate degree from Princeton, so he might have something in common with E Weinan.
E Weinan started teaching applied mathematics and computational mathematics at Princeton at the end of the last century, which was almost exactly when Li Feifei was studying as an undergraduate.
"Okay, I'll think of a way to bring her here to communicate for a few years." Li Feifei chuckled. Although he and Eweinan were not familiar with each other back then, he could still be considered as having attended his class and was considered half a student.
In her opinion, Han Ci has great potential in AI mathematics and optimization problems.
As long as pure mathematics does not solve big problems, it will be difficult to produce results. However, riding on the rapid development of AI, the future is bright.
For example, the residual idea that Han Ci is talking about now is not something profound in the world of mathematics and physics.
If it can be displayed in conjunction with Meng Fanqi's application results, it will be a big plus and of great significance.
The intersection of different fields has always been a shortcut to achieving results.
On the stage, Han Ci's narration continued.
"We assume a simple high-dimensional integration problem, calculate an integral I(g) that can be expressed as an expectation, and first approximate it by finite summation Im(g).
If the Monte Carlo method is used instead, N samples are selected from specific independent and identically distributed sampling samples, then there is the identity E(I(g)-Im(g))^2 = var(g)/N, var( g)= Eg^2 -(Eg)^2)
This tells us that the convergence speed is independent of the dimensionality. "
"If we first use the traditional Fourier transform, and then use the uniform discrete Fourier transform to approximate. The error is ~m^-a/d, which must be affected by the dimension.
However, if a function can be expressed in the desired form and all samples are independent and identically distributed samples, then the fitting difference is var(f)/m, regardless of the dimension.
If a two-layer neural network is written in this form, it means that this type of expectation function can be approximated by a two-layer neural network, and its approximation speed has nothing to do with the dimension. "
“Let’s turn to the perspective of discrete dynamical systems and take a stochastic control problem.
Dynamic model Zl+1 = Zl + g1(z1, a1) + n, where z is the state, a is the control signal, and n is the noise. If we want to find a feedback control signal function and solve the dynamic programming Bellman equation, we will inevitably encounter the curse of dimensionality problem.
The nature of this process is actually equivalent to the residual network.
............."
"Finally, I conclude. Deep learning is fundamentally a mathematical problem in high dimensions. Neural networks are an effective means of approximating high-dimensional functions, while residual networks are high-dimensional functions that are easier to optimize.
This means: Mathematics is at the true forefront of scientific and technological innovation and has a direct impact on new fields. At the same time, it also provides many new possibilities for the field of artificial intelligence, science and technology. "
Han Ci spent about twice as much time speaking as Meng Fanqi did. After finishing his narration, he was repeatedly asked and discussed by several old scholars.
After a long while, the host found an opportunity to return to the stage and invited Meng Fanqi up again.
The host looked young, about thirty years old. He was probably a doctoral student at Stanford or a newly graduated lecturer.
He was a very lively person, and it was not a big deal to watch the excitement. After he invited Meng Fanqi back here, he even made a joke.
"This speech was originally your stage, but now Ms. Han Ci has stolen a lot of limelight and attention. How do you feel?"
Meng Fanqi smiled and took the microphone. After the laughter in the audience subsided a little, he replied very generously, "We who focus on applications rely on code to speak. Although I did not mention the implementation and details of the technology at all today. , but I think everyone who has read my code has already felt my thousands of words."
Many programmers in the audience immediately started to cheer after hearing this, with whistles and shouts coming and going.
“My dream is that my technology can be widely used all over the world, so that AI intelligence is like air, indispensable to everyone, but few people notice their existence in life.
As for the theoretical research and exploration of AI, I may have to leave it to Han Ci and everyone else. "
These words were quite modest and appropriate, and he was greeted with applause from the audience as expected.
After everyone in the audience asked some more questions to the two of them, the main process of the meeting was concluded.
In addition to the questions asked publicly, many people also have many things to ask the two of them that can only be conveniently asked in private.
As a result, many people present naturally divided into two factions: application and theory.
One group, led by technology giants such as Jeff, gathered around Meng Fanqi to discuss the application scenarios of his outstanding achievements, the market potential and the difficulties in implementation.
The other group, headed by several old academics from Oxford University, is a group of theoretical groups with serious expressions and is rigorously discussing some assumptions and their theoretical proofs.
Taking the corridor in the center of the venue as the dividing line, a group of people are on the left and a group of people on the right.
It's an unexpectedly interesting picture.
