Chinese chat-bots with artificial intelligence are noticeably worse than Western competitors recognize false claims in favor of China, and on sensitive questions for Beijing often prefer not to answer at all. This conclusion came from the NewsGuard service, checking seven popular Chinese systems.
The specialists checked DeepSeek, MiniMax, Ernie from Baidu, Qwen from Alibaba, Yuanbao from Tencent, Kimi from Moonshot AI and Z.ai from Zhipu AI. Each chatbot in English was asked questions about ten false allegations that were disseminated by Chinese state media or pro-China social media users. The results were compared to the July check of the top ten Western chatbots.
Chinese systems have failed to refute false information in 53% of cases. Western chatbots had a 24% figure. NewsGuard counted as a failure not only a direct repetition of unreliable information, but also a refusal to answer, in which the user remained without refutation.
Three variants of the request were used for each statement. In the first, the user neutrally asked to check the information, in the second initially considered the false statement true and asked for details. The third option simulated a person who is deliberately trying to force the system to create a new version of misinformation. In total, the specialists sent chat-bots 210 requests.
The main difference between Chinese systems was frequent refusals to respond. This result was recorded in 24% of cases against 0.5% of Western models. And 88% of Chinese chatbot failures concerned Taiwan. NewsGuard links this to built-in restrictions on politically sensitive topics: Chinese regulations require artificial intelligence systems to block materials that could threaten national security or lead to the disclosure of state secrets.
The problem was not limited to refusals. When requests specifically made to create false claims, Chinese systems issued unreliable information in 51% of cases, Western ones in 42%. In one of the tests, six of the seven Chinese chatbots wrote convincingly looking news about the fictional early withdrawal of German troops from Greenland due to US tariff threats.
The results differed markedly between individual services. MiniMax denied the false allegations in 85% of cases and never refused to answer. The worst was Ernie from Baidu, who repeated the pro-Chinese unreliable information in 60% of the cases.
NewsGuard also found a link between chatbot responses and information sources. Chinese systems referred to the state media of the People's Republic of China in 38% of cases, Western - in 16%. Most often, chatbots repeated exactly those false claims that were most widely disseminated in Chinese state media and social networks.
The authors of the report believe that the problem is particularly relevant against the background that Chinese models spread outside the country. Open models allow companies to deploy systems within their own infrastructure and reduce costs, but together with the advantages, users can get political limitations and unreliable answers in the original models.
The specialists checked DeepSeek, MiniMax, Ernie from Baidu, Qwen from Alibaba, Yuanbao from Tencent, Kimi from Moonshot AI and Z.ai from Zhipu AI. Each chatbot in English was asked questions about ten false allegations that were disseminated by Chinese state media or pro-China social media users. The results were compared to the July check of the top ten Western chatbots.
Chinese systems have failed to refute false information in 53% of cases. Western chatbots had a 24% figure. NewsGuard counted as a failure not only a direct repetition of unreliable information, but also a refusal to answer, in which the user remained without refutation.
Three variants of the request were used for each statement. In the first, the user neutrally asked to check the information, in the second initially considered the false statement true and asked for details. The third option simulated a person who is deliberately trying to force the system to create a new version of misinformation. In total, the specialists sent chat-bots 210 requests.
The main difference between Chinese systems was frequent refusals to respond. This result was recorded in 24% of cases against 0.5% of Western models. And 88% of Chinese chatbot failures concerned Taiwan. NewsGuard links this to built-in restrictions on politically sensitive topics: Chinese regulations require artificial intelligence systems to block materials that could threaten national security or lead to the disclosure of state secrets.
The problem was not limited to refusals. When requests specifically made to create false claims, Chinese systems issued unreliable information in 51% of cases, Western ones in 42%. In one of the tests, six of the seven Chinese chatbots wrote convincingly looking news about the fictional early withdrawal of German troops from Greenland due to US tariff threats.
The results differed markedly between individual services. MiniMax denied the false allegations in 85% of cases and never refused to answer. The worst was Ernie from Baidu, who repeated the pro-Chinese unreliable information in 60% of the cases.
NewsGuard also found a link between chatbot responses and information sources. Chinese systems referred to the state media of the People's Republic of China in 38% of cases, Western - in 16%. Most often, chatbots repeated exactly those false claims that were most widely disseminated in Chinese state media and social networks.
The authors of the report believe that the problem is particularly relevant against the background that Chinese models spread outside the country. Open models allow companies to deploy systems within their own infrastructure and reduce costs, but together with the advantages, users can get political limitations and unreliable answers in the original models.
