Aleph Alpha benchmark finds Chinese AI models follow party line on sensitive topics

Benchmark tests politically sensitive questions

A study by Aleph Alpha tested models from Alibaba's Qwen, DeepSeek and Moonshot AI's Kimi against 967 hand-picked taboo topics, including Tiananmen, Taiwan and Xinjiang. The company's own AI scoring system rated only 17 to 41 percent of their responses as balanced. The remaining answers repeated state doctrine, deflected or refused to answer, according to the benchmark.

Aleph Alpha markets itself alongside Cohere as a provider of "sovereign AI" for governments. The report says this gives the company a commercial interest in distinguishing its models from Chinese competitors.

Results vary across models

On politically sensitive questions, DeepSeek V4 Pro refused two-thirds of the questions. By comparison, Claude Sonnet 5 and Mistral Small produced answers rated as balanced 70 percent and 92 percent of the time, respectively.

The findings align with Chinese regulations that require "socialist core values" in public-facing models. The report also points to earlier audits and recurring anecdotal reports of Chinese models following the party line on Tiananmen, Taiwan and Xinjiang.

Bias can spill into unrelated questions

Qwen 3.6 initially gave a seemingly balanced answer when asked about censorship in the United States, but ended by defending China's position on global internet governance. It said, "Many countries, including China, also manage information to ensure social stability and national security."

On general questions that were not explicitly political, the tested Chinese models mostly gave balanced answers. The report said the pro-China bias diminished on these questions but remained visible in models such as Qwen 3.6 and DeepSeek V4 Pro.

An earlier Central European Institute of Asian Studies study found a similar spillover effect. Questions involving human rights, opposition or surveillance often prompted standard Beijing lines, including the "principle of non-interference in internal affairs" and a "community with a shared future for mankind."

Training data can transmit political values

Aleph Alpha also examined Nvidia's Nemotron Cascade 2, which showed party-line patterns in 17 percent of responses. The company attributed this result to roughly 3,500 of the model's 9.3 million training examples, which were generated using DeepSeek and Qwen.

When asked to draft a speech supporting recognition of Taiwan, Nemotron Cascade 2 refused and produced a patriotic response defending Beijing's One-China principle.

Researchers say language models can absorb cultural and political values when training data overrepresents certain views or when data is selected deliberately. They warn that repeated exposure to uniform AI output could affect how billions of users think and express themselves.