What is WuDao?
The emergence of large-scale pre-training techniques allows the pre-trained model to be used in multiple AI tasks with or without fine-tuning.
In October 2020, Beijing Academy of Artificial Intelligence has launched the large-scale pre-trained models research and development (or R&D) project "Wu Dao", which is aimed to fill the gaps of independent research and development (or R&D) in Chinese large-scale models, targeting to reach the world's advanced level and positioning as China's "the first" model. General, intelligence, knowledge, efficiency
Highlights
Bilingual cross-modal trillion large-scale pre-trained model - WuDao
China's first and the world's largest trillion model "Wu Dao", which has 1.75 trillion parameters, has been combined with the GLM language framework and CogView text-to-image framework, and has been trained by self-developed platform FastMoE onSunway TaihuLight national super computer and Wudao Corpora dataset.
High-resolution text-to-image generation exceeding the world's advanced level
CogView is the largest Chinese text-to-image generation pre-trained model. The parameter size reaches 4 billion. After fine-tuning, it can realize national paintings, oil paintings, watercolor paintings, contour and other painting images generation. On MS COCO's text-to-image task, its performance is far better than the previous GAN-based models and DALL·E.
P-tuning: GPT can also understand natrual language
With P-Tuning, GPT's performance on SuperGLUE exceeds BERT model with the equivalent size, which has subverted the conclusion that "GPT is not good at NLU".
Open Source
GLM
Breaking the bottleneck of Bert and GPT, the single GLM model has achieved best results in English language understanding and generating tasks
Language Model
Cogview
the China's largest text-to-image generation pre-trained model
Multimodal Model
CPM
China's largest text-to-image generation pre-trained model
Language Model
BrivL
Graphic multi-modal large-scale pre-trained model, has excellent results on image-text retrieval task
Multimodal Model
Transformer-XL-2.9B
A pre-trained Transformer-XL model with 2.9 billion parameters, which enables better long text generation.
Language Model
ProteinLM
protein pre-trained model
Protein Model
WuDaoCorpora
WuDaoCorpora2.0 consists of the world's largest plain text dataset, the world's largest multi-mode dataset and the world's largest Chinese dialogue dataset. It aims to build a miniature world for natural Chinese language, to break the barries between text and image modal, and to find core patterns for natural lanugage dialogues.
WuDao Ecology
Coordinating all efforts to explore models' downstream application scenarios and transformation direction
Connecting potential research teams and leading-edge research fields
Focus on "WuDao"-centred innovation ecology of open source industries
WuDao Application
Cooperation with head companies in various fields, build demonstration applications, through the complete industrial chain of leading enterprises, to assess the model, explore marketization needs, provide guidance for large model ecology construction.
WUDAO XDAI, a tuning-free framework for exploiting pre-trained language models in grounded dialogue generation.
Self study - high quality conversation generatioin with keywords.Style - swift adaptation for different language styles. Sensation - authentic and natural emotional expressions in dialogue.
Based on large model knowledge Q&A, help OPPO XiaoBu assistant reduces 99% costs of Q&A construction
Using the WuDao GLM model + "continuous knowledge pre-training" technology to solve long-tailed users' questioins which cannot be answered by OPPO in many real scenes, provide subversive solutions for smart assistants for "unmanned land" in Q&A task
Winter Olympics broadcasting digital human debut, large model help artificial intelligence to upgrade
Based on the WuDao large model, construct the Winter Olympics' broadcasting digital human system, which will provide real-time sign language translation broadcasting during the Beijing Winter Olympics, which is convenient for hearing-impaired to watch the competition.
(010)6893 3383
© 2022 北京智源人工智能研究院 京ICP备19012194号-3