[Xinzhiyuan Guide] Ancient literati, or chant and chant, narrate quiet feelings, or wind and dance, chant and return. "Reciting Poems and Composing Pairs" became their standard. Just now, the world's largest artificial intelligence massive model "Source 1.0" was released.

Science students may have nothing to do with liberal arts students when they start to literary.

Do not believe? Look at this seven-character poem:

Although it is not the toad palace banished to immortal, why be afraid of the cold to the bones of the ice palace.

Peeping curtain squinted at Jinwu Xiao, how many talents are here.

After reading it, I have to say that it is really good beer! The connotation of artistic conception is very good.

Not only can write poems, but also write words, such as the following:

I suspected that there were tears in nine days,

Sprinkle it for me.

Drip into the water of West Lake,

Wet the moonlight a thousand miles away,

Turned into the cloud in my dream.

Can you imagine that this is a masterpiece of a science and engineering student who does not know how to write poetry at all?

It is true. Just let Li Bai readers will be silent, let Du readers will be tears .

This is the world's largest artificial intelligence massive model just released by Inspur, called "Source 1.0".

In addition to being able to compose poems, it can also write dialogues, write couplets, generate news, continue writing stories...

With 245.7 billion parameters , the world's largest artificial intelligence model has read 200 billion words.

You know, there is no way to read so many words in a person's life.

Since it is called the largest in the world, how big is it?

The world's largest artificial intelligence massive model!

The title of the world's largest is not a joke!

"Source 1.0" has achieved ultra-large scale and huge quantification in terms of algorithms, data, or computing power.

In terms of algorithms, compared to the English language model GPT-3 with 175 billion parameters, "Source 1.0" contains a total of 245.7 billion parameters, which is 1.404 times the amount of the former .

Moreover, the most important thing is that "Source 1.0" is a single model like GPT-3, instead of being piled up by many small models. In this aspect alone, "Source 1.0" can rank as the world's largest natural language understanding model.

In terms of data, "Source 1.0" has almost "read" all the vast contents of the entire Chinese Internet in the past five years . Through the self-developed text classification model, a 5TB high-quality Chinese data set was obtained , which is nearly 10 times ahead of the training data set size.

In addition, "Source 1.0" has read about 200 million words . what is this concept?

If a person can read ten books a month, a hundred books a year, and 50 years of reading, he will read 5,000 books in his lifetime. If a book is 200,000 words, it will only add up to 1 billion words, and it will take 10,000 years to read. Over 200 billion words . With the support of such a large-scale data, the data set of "Source 1.0" has naturally become the world's largest high-quality Chinese data set.

In terms of computing power, "Source 1.0" consumes a total of about 4095PD (PetaFlop/s-day) . Compared with GPT-3 consumes 3640PD calculations to obtain 175 billion parameters, the calculation efficiency is greatly improved. If you let "Source 1.0" "read" 24 hours a day, it only takes 16 days to read almost all the contents of the Chinese Internet in the past five years .

Um? Have you learned all the Chinese Internet content for the past 5 years?

u1s1, "source 1.0" yyds!

What are the results?

Is it fun to experience the "Turing Test" for yourself? Don't worry, there are still many questions waiting for you!

Now let's take a look at the most powerful imitation ability of "Source 1.0". That is, for a "word" that does not exist at all, only one use case is needed, "Source 1.0" can learn the usage of this "word".

Seeing these familiar "words" do you feel that smell (doge) ? Suddenly I was looking forward to it. If "Source 1.0" learns how to use the word "The clown is myself" , hehe.

Now that the Turing test is mentioned, let's take a look at the results of the test.

"Source 1.0" achieved an average false positive rate of up to 50.84% in the test !

The Turing test adopts the "question" and "answer" modes, that is, the observer controls the typewriter to talk to two test subjects, one of which is a human and the other is a machine. The observer keeps asking various questions to distinguish whether the answerer is a human or a machine.

It is generally believed that after multiple tests, if the machine makes each participant make more than 30% misjudgments, then the machine has passed the test and is considered to have human intelligence.

In the test result of "Source 1.0", the average discrimination correct rate of the respondents was 49.16%, which means the average false positive rate was 50.84%. In the field of news generation, the misjudgment rate is as high as 57.88%.

Among them, the correct rate refers to the proportion of the text generated by the machine can be correctly recognized. The correct rate is less than 70%, which means that the false positive rate is more than 30%.

The data looks too monotonous? fine! The "real questions" session that everyone is looking forward to is here!

What is the experience of being No. 1 in the world?

So, how strong is this largest AI model in the world?

It's better to pull out and run a score and check the list!

English language model evaluation includes GLUE and SuperGLUE. Various pre-trained models such as GPT-3 will be evaluated on it. Similar to GLUE, CLUE is the first large-scale language evaluation benchmark for Chinese. It includes representative data sets, benchmark (pre-training) models, corpora, and rankings. And these data sets will also cover different tasks, data volume, task difficulty, etc.

By the way, Amway recently released the first data-centric AI evaluation DataCLUE in China.

Closer to home, "Source 1.0" occupies the top of the two lists of zero-shot and few-shot.

In the ZeroCLUE zero-sample learning list, "Source 1.0" is far ahead with an absolute advantage of 18.3% surpassing the industry's best score. Among them, he won the championship in the 6 tasks of document classification, news classification, product classification, native Chinese reasoning, idiom reading comprehension, and noun pronoun relations.

In the FewCLUE small sample learning list, "Source 1.0" won the champion of 4 tasks including document classification, product classification, document abstract recognition, and noun pronoun relations.

Zero-sample learning means that the trained classifier can not only identify the existing data categories in the training set, but also distinguish data from unseen categories. In principle, it is to make the computer have human reasoning and knowledge transfer ability, without any training data to be able to recognize a new thing that has never been seen before.

Small sample learning is to use much less than the amount of data samples required for deep learning to achieve an effect close to or even surpassing big data deep learning. Whether it has the ability to learn and generalize from a small number of samples is an obvious dividing point between artificial intelligence and human intelligence. Because humans can easily build awareness of new things with only one or a few examples, machine learning algorithms usually require thousands of supervised samples to ensure their generalization ability.

After talking for a long time, what's the use of the small sample learning and zero sample learning of "Source 1.0"?

This is about to mention a very important meaning of the massive model : powerful unified generalization capabilities.

For most models with relatively small scales, fine-tuning is required for each new task, and the corresponding data set is fed to it. After a lot of work has been done, it can be applied in new scenarios. For a large number of models, when faced with different application tasks, there is no need to do a lot of retraining and readjustment.

Wu Shaohua, the chief researcher of Inspur Artificial Intelligence Research Institute, said: "You don't need to feed a huge amount of data for training, and you can get very good results in a new application scenario."

Therefore, the adaptability of the massive model is very strong, which can greatly reduce the industry's investment in data or fine-tuning when applying the model, thereby accelerating the development of the industry.

How to evaluate?

Large models are becoming the trend of AI development and are a must-see high ground.

Time has to go back three years ago... The pre-trained model at that time successfully activated the deep neural network and the self-supervision capabilities of large-scale unlabeled data.

The switch of deep learning model and performance is turned on at the same time, especially in the field of NLP.

After Big Tech has tasted the benefits of training models, they have launched fierce competitions for model scale and performance.

From the stunning four-seater Google BERT to OpenAI's GPT-3, the amount of parameters is constantly refreshed, with 175 billion parameters, and its capabilities are self-evident.

At present, the training of language models has moved from a "large model" to a "large model" stage, and a large number of models have also become the focus of attention in the industry.

Recently, Li Feifei and other Stanford researchers explained in the paper that the significance of the quasi-mass model lies in emergence and homogeneity. In the paper, they gave this large model a name, called the foundation model, and systematically discussed the opportunities and risks of the foundation model.

Simply put, the big model is our understanding of the evolution of life, from simple to complex.

We compare the model to the life in the meta-universe. How large it has the capabilities of this complex integrated system may determine the level of intelligence in the digital world and the intelligent world in the future.

Today, "Source 1.0" has 245.7 billion parameters that are not enough. Human neuronal synapses exceed 100 trillion, so there is still a long way to go.

And where is the innovation of "Source 1.0"? Through collaborative optimization, "Source 1.0" overcomes industry problems in terms of scalability, computational efficiency, massive model algorithms, and precision enhancement of huge amounts of data and ultra-large-scale distributed training.

Algorithmically:

1. Solved the industry problem of unstable training of huge models, and proposed an algorithm for stable training of huge models;
2. A new reasoning method for massive models is proposed to improve the generalization ability of the model, so that a model can be applied to more scenarios.

Data:

An innovative method for generating Chinese data sets is proposed. Through a brand-new text classification model, junk text can be effectively filtered and high-quality Chinese data sets can be generated.

In terms of computing power:

"Source 1.0" through the collaborative optimization of algorithms and computing power, makes the model more conducive to GPU performance, greatly improves computing efficiency, and achieves industry-leading training performance while achieving industry-leading accuracy.

So, what can developers get from this "black land"?

The Inspur 1.0 large model is just the beginning, it just provides a vast fertile soil.

In the future, Inspur will open its large-scale model APIs to serve all developers in the Yuannao ecological community for developers around the world to develop applications on the platform for all walks of life.

Various applications can perform large model-based search, dialogue, text completion and other advanced AI functions through the API provided by Inspur.

In fact, whether it is 175 billion parameters or 245.7 billion huge parameter language models, the most important thing is whether it can really be used by us. When it comes to playing, the real meaning is not the first show at the press conference, but the end to play its role and value in the actual scene.

Liu Jun, vice president of Inspur Information, said, "First of all, from the birth of the large model itself, there is another meaning, that is, for the exploration of cutting-edge technology, a platform such as a large model is needed to support further development on this platform. Innovation."

"Secondly, in the industry, many of our industry representatives have put forward killer application scenarios, such as operator smart operation and maintenance, automatic generation of reports in smart office scenarios, and automatic dialogue with smart assistants."

The "Source 1.0" large model has the ability to "recognize themes and generate summaries" from natural language, allowing the product, customer experience and marketing teams of companies in all walks of life to better understand the needs of customers.

For example, the future big model identifies topics and emotions from surveys, service desk tickets, real-time chat logs, comments, etc., and then extracts insights from this aggregated feedback and provides a summary within a few seconds.

If asked "What makes our customers frustrated with the checkout experience?"

A large model might provide insights like: "Customers are frustrated with the checkout process because it takes too long to load. They also want a way to edit the address and save multiple payment methods during checkout."

In the future, the Inspur 1.0 large model will promote innovative companies and individual developers to build more intelligent scenarios based on the large model, empower the intelligent upgrade of the real economy, and promote high-quality economic development.

Turing test answer

dialogue

couplet

Poetry