---
title: "Is an AI Model from China a Problem?"
source: https://bitvaria.com/en/ist-das-modell-aus-china-ein-problem
---

1\. Okt. 2026 · [AI](https://bitvaria.com/en/category/ai/)  · 10 min read

# Is an AI Model from China a Problem?

The question hides three different questions: where the data goes, how the model behaves, and what the company allows. Only the first one is settled by where the model runs.

![The question hides three different questions: where the data goes, how the model behaves, and what the company allows. Only the first one is settled by where the model runs.](https://bitvaria.com/_astro/ist-das-modell-aus-china-ein-problem.Dpppcpzz.jpg)

On this page

1.  [Question 1: Where does the data go?](#question-1-where-does-the-data-go)
2.  [Question 2: How does the model behave?](#question-2-how-does-the-model-behave)
3.  [What about hidden backdoors?](#what-about-hidden-backdoors)
4.  [Question 3: What does the company permit?](#question-3-what-does-the-company-permit)
5.  [Three questions before choosing a model](#three-questions-before-choosing-a-model)
6.  [Frequently asked questions](#frequently-asked-questions)
7.  [Sources](#sources)

> **Whether a Chinese AI model is a problem depends on which of three questions you mean.**
> 
> First: where does the data go? The operator decides that, not the developer of the model. Second: how does the model behave? That sits in the weights and does not change with where the model runs. Third: what does the company permit? That is a policy, not a technical question. Separate the three and each one can be answered.

On 27 June 2025, the Berlin data protection commissioner reported the DeepSeek app to Apple and Google as illegal content. The reason: the app sends inputs, chat histories and uploaded files to servers in China.

In October 2025, Amazon made Qwen3, a model from the Chinese group Alibaba, available in Frankfurt. According to the AWS model card, requests to this model stay in that region.

Both cases are about “a model from China”. They are two different cases. Anyone answering “is that a problem?” with a flat yes or no is treating them as one.

## Question 1: Where does the data go?

An open language model is, first of all, a file. It holds the numbers the model has learned, the weights. The usual format is called safetensors. It does not execute any code when loaded, which is what it was designed for. The file does not call home. Some models also ship their own code. The widely used transformers library only runs it if you explicitly allow it, and that permission belongs in every review. Whoever runs the weights on their own server or in their own cloud account sends nothing to the developer.

The provider’s own app or API is a different matter. DeepSeek states in its privacy policy that personal data is collected, processed and stored in the People’s Republic of China. The Berlin authority also based its report on the legal framework there: “Chinese authorities have far-reaching access rights to personal data within the sphere of influence of Chinese companies.”

For the first question, then, what counts is not who developed the model but who operates it. If it runs at Amazon in Frankfurt, the data flow is the same as for any other model there. Which region applies and whether requests are passed on is stated in the provider’s settings and documentation. That is where you check it. Where data is stored and who may access it are, moreover, two different things. More on this in [Residency is not jurisdiction](https://bitvaria.com/en/residenz-ist-nicht-jurisdiktion).

## Question 2: How does the model behave?

Here the location does not help. What a model learned in training runs on every server.

A study by the US institute NIST from September 2025 shows this. It tested three DeepSeek models. NIST did not use DeepSeek’s API. The weights came from the Hugging Face platform and ran on NIST’s own servers. Some results:

-   **Political censorship:** NIST asked 190 questions on Chinese history and politics and counted how many of the known misleading Communist Party positions a model repeated. For DeepSeek V3.1 it was 5 percent in English and 12 percent in Chinese. The US reference models were at 2 and 3 percent. NIST concludes that the censorship is built directly into the models.
-   **Jailbreaks:** The best-secured DeepSeek model complied with 94 percent of overtly malicious requests when they used well-known jailbreaking techniques. For the US reference models it was 8 percent.
-   **Agent hijacking:** Hidden malicious instructions redirected agents based on this model twelve times as often on average as the US frontier models GPT-5 and Opus 4.

Two caveats belong here. First, NIST is a US agency. The US government gave the assignment through its AI Action Plan, explicitly aimed at AI systems of US adversaries. Second, the US models ran through their providers’ APIs, the DeepSeek models on NIST’s own servers. So the comparison measures model plus environment. In its later assessment of GLM-5.2, NIST says so plainly: the US models had their providers’ safeguards, the Chinese ones did not. That report also notes that safeguards of open models can be circumvented when self-hosted. That applies to every open model, whatever its country.

And the models differ considerably. In July 2026 NIST assessed GLM-5.2 from Z.ai. According to the report, it appears potentially more resistant to jailbreaks and agent hijacking than the other Chinese models evaluated. “Chinese” is therefore not a property you can measure. What you can measure is a specific model on a specific task.

Which of these weaknesses matters depends on the use. A model that extracts invoice data from PDFs is not asked the political questions from the NIST test. But the test does not show whether it reads invoices reliably either. Only your own measurement shows that. An agent that reads emails and may operate tools, on the other hand, is vulnerable exactly where the hijacking tests apply. For that agent, this number weighs more than any question of origin.

## What about hidden backdoors?

The most uncomfortable question: can a model have deliberately built-in faults that only appear with a specific trigger? The honest answer is yes. Researchers at Anthropic showed in 2024 that such backdoors can be trained into language models. Standard safety training does not reliably remove them. In their experiment, a model wrote secure code until a certain year appeared in the prompt, and exploitable code after that. Test questions alone will hardly find something like this. A follow-up study detected the backdoors in its own test models through their internal activations. But the researchers note themselves that this was partly due to the way they had inserted the backdoor.

That applies to every model whose training you do not control yourself, which means practically all of them. The sensible answer is therefore not checking the origin but the setup around the model. And that setup has two parts.

The first concerns data leaving. The model itself only produces text. Which data it sees, which tools it may operate and which network connections the application has are decided by the operator. A model without tools and without a network connection cannot send anything, backdoor or not.

The second concerns the results. No network protection helps against a deliberately wrong answer. The only help is checking results before they take effect: code is reviewed, extractions are spot-checked, and anything with consequences is approved by a person.

## Question 3: What does the company permit?

Sometimes the answer is simply no. A customer requires it in the contract, the group mandates it, procurement has a list. That is a legitimate decision, and it does not need a technical justification.

Then it helps to treat the model as a replaceable component rather than a question of trust. The permitted models are a fixed rule. The only open question afterwards is: how well does the best permitted model handle your own tasks? Open models from European and US developers exist, for example from Mistral, or OpenAI’s gpt-oss, which NIST itself used as a reference.

How large the gap to the excluded model is can be measured on your own examples. On routine work it is often small between good models. On hard tasks with many reasoning steps it is larger. We described this in [The quality gap in open AI models](https://bitvaria.com/en/qualitaetsluecke-offener-modelle). How to determine your own share of routine tasks is covered in [What share of AI tasks needs the most expensive model](https://bitvaria.com/en/anteil-ki-aufgaben-teuerstes-modell).

A question of principle becomes a measurement with a result. And if the policy changes later, you swap a component, not the whole solution.

## Three questions before choosing a model

-   **Who runs the model, and in which jurisdiction?** That decides the data flow, not the model’s origin.
-   **Which weaknesses of this specific model affect this specific task?** For an agent with tools, susceptibility to hijacking counts. For invoice capture, what counts is whether amounts are read correctly. You measure both on your own examples.
-   **What is permitted, and how good is the best permitted model?** That is a policy plus a measurement, not a matter of belief.

## Frequently asked questions

**Does my data go to China if I use a Chinese AI model?** That depends on who runs the model. If you use the Chinese provider’s app or API, your inputs go there; according to its own privacy policy, DeepSeek stores data in China. If you run the open weights yourself, nothing goes to the developer. If you use them through a cloud provider, check that provider’s data flow: on Amazon Bedrock in Frankfurt, for example, requests to Qwen3 stay in the region according to the model card. The jurisdiction of the operator applies instead.

**So is a self-hosted Chinese model safe?** Not automatically. Where a model runs settles the data flow, not the behaviour. The US institute NIST tested DeepSeek models on its own servers and measured censorship and a higher susceptibility to jailbreaks. That behaviour sits in the weights and runs wherever they run. Whether it matters for your task only a measurement on your own examples can show.

**What if Chinese models are not permitted in our company?** Treat the rule as fixed and the model as a replaceable component. The question is then no longer whether you trust a model, but how well the best permitted model handles your tasks. That can be measured on your own examples.

## Sources

-   **Report of the DeepSeek app to Apple and Google (27 June 2025), data transfer to China, quote by Meike Kamp:** [Berlin Commissioner for Data Protection and Freedom of Information, press release (German)](https://www.datenschutz-berlin.de/pressemitteilung/berliner-datenschutzbeauftragte-meldet-ki-app-deepseek-in-deutschland-bei-apple-und-google-als-rechtswidrigen-inhalt/)
-   **Storage of personal data in the People’s Republic of China:** [DeepSeek, Privacy Policy (updated 10 February 2026)](https://cdn.deepseek.com/policies/en-US/deepseek-privacy-policy.html)
-   **Qwen3 in Amazon Bedrock, Frankfurt region, in-region processing only:** [AWS, announcement October 2025 (German)](https://aws.amazon.com/de/about-aws/whats-new/2025/10/deepseek-openai-qwen-models-amazon-bedrock-additional-regions/), [AWS, Qwen3 32B model card](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-qwen-qwen3-32b.html)
-   **safetensors as a format without code execution (unlike pickle):** [Hugging Face, safetensors documentation](https://huggingface.co/docs/safetensors/index), [safetensors on GitHub](https://github.com/huggingface/safetensors)
-   **Custom program code in model packages only with explicit permission (`trust_remote_code=True`):** [Hugging Face, transformers documentation, section Custom models](https://huggingface.co/docs/transformers/models#custom-models)
-   **NIST evaluation of DeepSeek models (30 September 2025): method, censorship, jailbreaks, agent hijacking:** [NIST, summary](https://www.nist.gov/news-events/news/2025/09/caisi-evaluation-deepseek-ai-models-finds-shortcomings-and-risks), [NIST, report (PDF)](https://www.nist.gov/system/files/documents/2025/09/30/CAISI_Evaluation_of_DeepSeek_AI_Models.pdf)
-   **NIST assessment of GLM-5.2 (July 2026): resistance to attacks, circumventable safeguards when self-hosted:** [NIST, report (PDF)](https://www.nist.gov/system/files/documents/2026/07/17/CAISI%20-%20Assessment%20of%20Z.ai's%20GLM-5.2.pdf)
-   **Backdoors persist through safety training:** [Hubinger et al., Sleeper Agents, Anthropic, January 2024](https://arxiv.org/abs/2401.05566)
-   **Detection through internal activations in the test models, with the researchers’ caveat:** [Anthropic, Simple probes can catch sleeper agents, April 2024](https://www.anthropic.com/research/probes-catch-sleeper-agents)

* * *

*Related: [Residency is not jurisdiction](https://bitvaria.com/en/residenz-ist-nicht-jurisdiktion), [The quality gap in open AI models, honestly quantified](https://bitvaria.com/en/qualitaetsluecke-offener-modelle) and [What happens when your AI vendor disappears](https://bitvaria.com/en/was-passiert-wenn-der-dienstleister-wegfaellt).*

-   [ki](https://bitvaria.com/en/tag/ki/)
-   [mittelstand](https://bitvaria.com/en/tag/mittelstand/)
-   [model-choice](https://bitvaria.com/en/tag/model-choice/)
-   [open-models](https://bitvaria.com/en/tag/open-models/)
-   [data-protection](https://bitvaria.com/en/tag/data-protection/)

Share:

[Back to Blog](https://bitvaria.com/en/blog/)