[Xinzhiyuan Guide] Ancient literati, or chant and chant, narrate quiet feelings, or wind and dance, chant and return. "Reciting Poems and Composing Pairs" became their standard. Just now, the world's largest artificial intelligence massive model "Source 1.0" was released.

Science students may have nothing to do with liberal arts students when they start to literary.

Do not believe? Look at this seven-character poem:

Although it is not the toad palace banished to immortal, why be afraid of the cold to the bones of the ice palace.

Peeping curtain squinted at Jinwu Xiao, how many talents are here.

After reading it, I have to say that it is really good beer! The connotation of artistic conception is very good.

Not only can write poems, but also write words, such as the following:

I suspected that there were tears in nine days,

Sprinkle it for me.

Drip into the water of West Lake,

Wet the moonlight a thousand miles away,

Turned into the cloud in my dream.

Can you imagine that this is a masterpiece of a science and engineering student who does not know how to write poetry at all?

It is true. Just let Li Bai readers will be silent, let Du readers will be tears .

This is the world's largest artificial intelligence massive model just released by Inspur, called "Source 1.0".

In addition to being able to compose poems, it can also write dialogues, write couplets, generate news, continue writing stories...

With 245.7 billion parameters , the world's largest artificial intelligence model has read 200 billion words.

You know, there is no way to read so many words in a person's life.

Since it is called the largest in the world, how big is it?

The world's largest artificial intelligence massive model!

The title of the world's largest is not a joke!

"Source 1.0" has achieved ultra-large scale and huge quantification in terms of algorithms, data, or computing power.

In terms of algorithms, compared to the English language model GPT-3 with 175 billion parameters, "Source 1.0" contains a total of 245.7 billion parameters, which is 1.404 times the amount of the former .

Moreover, the most important thing is that "Source 1.0" is a single model like GPT-3, instead of being piled up by many small models. In this aspect alone, "Source 1.0" can rank as the world's largest natural language understanding model.

In terms of data, "Source 1.0" has almost "read" all the vast contents of the entire Chinese Internet in the past five years . Through the self-developed text classification model, a 5TB high-quality Chinese data set was obtained , which is nearly 10 times ahead of the training data set size.

In addition, "Source 1.0" has read about 200 million words . what is this concept?

If a person can read ten books a month, a hundred books a year, and 50 years of reading, he will read 5,000 books in his lifetime. If a book is 200,000 words, it will only add up to 1 billion words, and it will take 10,000 years to read. Over 200 billion words . With the support of such a large-scale data, the data set of "Source 1.0" has naturally become the world's largest high-quality Chinese data set.

In terms of computing power, "Source 1.0" consumes a total of about 4095PD (PetaFlop/s-day) . Compared with GPT-3 consumes 3640PD calculations to obtain 175 billion parameters, the calculation efficiency is greatly improved. If you let "Source 1.0" "read" 24 hours a day, it only takes 16 days to read almost all the contents of the Chinese Internet in the past five years .

Um? Have you learned all the Chinese Internet content for the past 5 years?

u1s1, "source 1.0" yyds!

What are the results?

Is it fun to experience the "Turing Test" for yourself? Don't worry, there are still many questions waiting for you!

Now let's take a look at the most powerful imitation ability of "Source 1.0". That is, for a "word" that does not exist at all, only one use case is needed, "Source 1.0" can learn the usage of this "word".

Seeing these familiar "words" do you feel that smell (doge) ? Suddenly I was looking forward to it. If "Source 1.0" learns how to use the word "The clown is myself" , hehe.

Now that the Turing test is mentioned, let's take a look at the results of the test.

"Source 1.0" achieved an average false positive rate of up to 50.84% in the test !

The Turing test adopts the "question" and "answer" modes, that is, the observer controls the typewriter to talk to two test subjects, one of which is a human and the other is a machine. The observer keeps asking various questions to distinguish whether the answerer is a human or a machine.

It is generally believed that after multiple tests, if the machine makes each participant make more than 30% misjudgments, then the machine has passed the test and is considered to have human intelligence.

In the test result of "Source 1.0", the average discrimination correct rate of the respondents was 49.16%, which means the average false positive rate was 50.84%. In the field of news generation, the misjudgment rate is as high as 57.88%.

Among them, the correct rate refers to the proportion of the text generated by the machine can be correctly recognized. The correct rate is less than 70%, which means that the false positive rate is more than 30%.

The data looks too monotonous? fine! The "real questions" session that everyone is looking forward to is here!

What is the experience of being No. 1 in the world?

So, how strong is this largest AI model in the world?

It's better to pull out and run a score and check the list!

English language model evaluation includes GLUE and SuperGLUE. Various pre-trained models such as GPT-3 will be evaluated on it. Similar to GLUE, CLUE is the first large-scale language evaluation benchmark for Chinese. It includes representative data sets, benchmark (pre-training) models, corpora, and rankings. And these data sets will also cover different tasks, data volume, task difficulty, etc.

By the way, Amway recently released the first data-centric AI evaluation DataCLUE in China.

Closer to home, "Source 1.0" occupies the top of the two lists of zero-shot and few-shot.

In the ZeroCLUE zero-sample learning list, "Source 1.0" is far ahead with an absolute advantage of 18.3% surpassing the industry's best score. Among them, he won the championship in the 6 tasks of document classification, news classification, product classification, native Chinese reasoning, idiom reading comprehension, and noun pronoun relations.

In the FewCLUE small sample learning list, "Source 1.0" won the champion of 4 tasks including document classification, product classification, document abstract recognition, and noun pronoun relations.

Zero-sample learning means that the trained classifier can not only identify the existing data categories in the training set, but also distinguish data from unseen categories. In principle, it is to make the computer have human reasoning and knowledge transfer ability, without any training data to be able to recognize a new thing that has never been seen before.

Small sample learning is to use much less than the amount of data samples required for deep learning to achieve an effect close to or even surpassing big data deep learning. Whether it has the ability to learn and generalize from a small number of samples is an obvious dividing point between artificial intelligence and human intelligence. Because humans can easily build awareness of new things with only one or a few examples, machine learning algorithms usually require thousands of supervised samples to ensure their generalization ability.

After talking for a long time, what's the use of the small sample learning and zero sample learning of "Source 1.0"?

This is about to mention a very important meaning of the massive model : powerful unified generalization capabilities.

For most models with relatively small scales, fine-tuning is required for each new task, and the corresponding data set is fed to it. After a lot of work has been done, it can be applied in new scenarios. For a large number of models, when faced with different application tasks, there is no need to do a lot of retraining and readjustment.

Wu Shaohua, the chief researcher of Inspur Artificial Intelligence Research Institute, said: "You don't need to feed a huge amount of data for training, and you can get very good results in a new application scenario."

Therefore, the adaptability of the massive model is very strong, which can greatly reduce the industry's investment in data or fine-tuning when applying the model, thereby accelerating the development of the industry.

How to evaluate?

Large models are becoming the trend of AI development and are a must-see high ground.

Time has to go back three years ago... The pre-trained model at that time successfully activated the deep neural network and the self-supervision capabilities of large-scale unlabeled data.

The switch of deep learning model and performance is turned on at the same time, especially in the field of NLP.

After Big Tech has tasted the benefits of training models, they have launched fierce competitions for model scale and performance.

From the stunning four-seater Google BERT to OpenAI's GPT-3, the amount of parameters is constantly refreshed, with 175 billion parameters, and its capabilities are self-evident.