Yandex publishes YaLM 100B, the largest GPT-like neural network in open source

Yandex has published YaLM 100B, a bilingual neural network for generating and processing text. It’s the largest GPT-like model in the public domain. It can be used freely by developers and researchers from all over the world.

YaLM 100B contains 100 billion parameters, more than any GPT-like model available in open-source under a license that allows commercial use. Researchers and developers can use the corporate-size solution to solve the most complex problems associated with natural language processing.

Language models from the YaLM family determine the principle of constructing a text and generate new ones based on the rules of linguistics and their knowledge of the world. YaLM can create texts, from poems to answers, and classify them according to the style of speech.

Yandex uses YaLM neural networks in over 20 projects, including its search engine and voice assistant Alice. Language models help support staff respond to requests and generate advertisements and site descriptions. YaLM neural networks also prepare quick answers in Yandex Search.

“Training generative neural networks requires substantial resources, experienced professionals and years of work. And it is important for us that not only the largest IT companies have access to modern technologies, but the entire community of researchers and developers. By making YaLM 100B publicly available, we hope to give impetus to further developing generative neural networks,” said Petr Popov, CEO of Yandex Technologies.

During the training, YaLM 100B processed about two terabytes of text from The Pile dataset and texts from the Internet. Yandex has published the model under the Apache 2.0 open license and posted it on GitHub.

Contacts:
Ilya Grabovskiy
Phone: +7 495 739-70-00
E-mail: pr@yandex-team.ru