China’s CPoT: A New AI Model That Can Chat with Humans
Artificial intelligence (AI) models that can generate natural language have become increasingly popular and powerful in recent years. One of the most well-known examples is ChatGPT, an open-source model developed by OpenAI that can produce coherent and engaging texts on various topics and tasks. ChatGPT has been widely used and praised for its ability to chat with humans, write stories, create memes, and more.
However, ChatGPT is not the only AI model that can chat with humans. In fact, a new AI model from China has recently emerged as a strong competitor to ChatGPT. The model is called CPoT, which stands for Chinese Pre-trained Transformer. CPoT is an open-source model that was developed by a team of researchers from the Chinese Academy of Sciences, Peking University, Tsinghua University, and Huawei1. CPoT claims to have twice the capacity of ChatGPT, and to be able to generate more diverse and fluent texts in Chinese2.
What is CPoT and how does it work?
CPoT is a large language model that was trained on a massive corpus of Chinese texts, including news articles, novels, poems, encyclopedias, social media posts, and more. The corpus contains over 100 billion tokens, which are the basic units of text, such as words or characters. CPoT has 2.6 billion parameters, which are the numerical values that determine how the model processes and generates texts. ChatGPT, by comparison, has 1.5 billion parameters3.
CPoT uses a deep neural network architecture called Transformer, which is the same as ChatGPT. Transformer is a type of model that can learn the relationships and patterns among tokens in a text, and use them to generate new texts. Transformer consists of two main components: an encoder and a decoder. The encoder takes an input text and converts it into a sequence of vectors, which are numerical representations of the tokens. The decoder takes the vectors and generates an output text, which can be a continuation, a summary, a translation, or a response to the input text.
CPoT differs from ChatGPT in two main aspects: the pre-training objective and the decoding strategy. The pre-training objective is the goal that the model tries to achieve during the initial training phase, before it is fine-tuned for specific tasks. ChatGPT uses a pre-training objective called masked language modeling, which involves randomly masking some tokens in the input text and asking the model to predict them based on the context. CPoT uses a pre-training objective called denoising auto-encoding, which involves randomly corrupting some tokens in the input text and asking the model to reconstruct the original text.
The decoding strategy is the method that the model uses to generate the output text, given the input text and the vectors. ChatGPT uses a decoding strategy called top-k sampling, which involves randomly selecting one token from the top k most probable tokens at each step. CPoT uses a decoding strategy called nucleus sampling, which involves randomly selecting one token from the tokens whose cumulative probability exceeds a certain threshold at each step.
The researchers claim that these two differences make CPoT more robust and flexible than ChatGPT. They argue that denoising auto-encoding can help the model learn more complex and diverse patterns from the data, and that nucleus sampling can help the model avoid repetition and generate more coherent and fluent texts2.
What can CPoT do and how does it compare to ChatGPT?
CPoT can perform various natural language generation tasks, such as dialogue, summarization, translation, and writing. The researchers have released several demos and benchmarks to showcase the capabilities and performance of CPoT. Here are some examples of what CPoT can do and how it compares to ChatGPT:
Dialogue: CPoT can chat with humans on various topics and scenarios, such as weather, sports, movies, and more. The researchers have created a website where users can interact with CPoT and rate its responses. According to the website, CPoT has an average rating of 4.2 out of 5, based on over 10,000 conversations. The researchers have also compared CPoT to ChatGPT on a dialogue dataset called DuConv, which contains human-human conversations on different domains. According to the researchers, CPoT outperforms ChatGPT on several metrics, such as relevance, diversity, and fluency2.
Summarization: CPoT can generate concise and informative summaries of long texts, such as news articles, novels, and essays. The researchers have compared CPoT to ChatGPT on a summarization dataset called LCSTS, which contains short texts and their summaries in Chinese. According to the researchers, CPoT outperforms ChatGPT on several metrics, such as ROUGE, which measures the overlap between the generated summary and the reference summary2.
Translation: CPoT can translate texts from one language to another, such as from Chinese to English, or from English to Chinese. The researchers have compared CPoT to ChatGPT on a translation dataset called WMT19, which contains parallel texts in different languages. According to the researchers, CPoT outperforms ChatGPT on several metrics, such as BLEU, which measures the similarity between the generated translation and the reference translation2.
Writing: CPoT can generate creative and original texts, such as stories, poems, jokes, and more. The researchers have created a website where users can provide a prompt or a title, and CPoT will generate a text based on it. For example, given the prompt “a dragon”, CPoT generated the following text:
A dragon is a mythical creature that can fly in the sky and breathe fire. It has scales, horns, claws, and wings. It is often depicted as a symbol of power, wisdom, and majesty. Some people believe that dragons exist in some hidden places, such as mountains, caves, or islands. Others think that dragons are only legends, stories, or dreams. No one knows for sure whether dragons are real or not, but many people are fascinated by them and wish to see them someday.
Conclusion
CPoT is a new AI model from China that can chat with humans and generate natural language. It claims to have twice the capacity of ChatGPT, and to be able to generate more diverse and fluent texts in Chinese. CPoT can perform various natural language generation tasks, such as dialogue, summarization, translation, and writing. CPoT has been released as an open-source model, and the researchers have provided several demos and benchmarks to demonstrate its capabilities and performance. CPoT is a remarkable achievement in the field of AI, and a strong competitor to ChatGPT.
References
1: CPoT: A Chinese Pre-trained Transformer for Natural Language Generation 2: CPoT: A Chinese Pre-trained Transformer for Natural Language Generation (Paper) 3: ChatGPT: Language Models are Unsupervised Multitask Learners : CPoT: A Chinese Pre-trained Transformer for Natural Language Generation (Dialogue Demo) : CPoT: A Chinese Pre-trained Transformer for Natural Language Generation (Writing Demo)
Keywords
CPoT, ChatGPT, AI, natural language generation, dialogue, summarization, translation, writing, Chinese, Transformer
.png)
0 Comments