In a surprising turn of events, a new player in the world of artificial intelligence has sent shockwaves through the tech industry and captured the imagination of millions. DeepSeek, a Chinese startup, has catapulted to the top of the app store charts with its groundbreaking AI assistant, DeepSeek R1. With an impressive 3 million downloads since its launch last week, DeepSeek has quickly outpaced its rivals, including the widely popular ChatGPT. What sets DeepSeek apart is not just its superior performance, but the remarkable efficiency with which it was developed. Using lower-powered chips and ingenious workarounds to navigate US sanctions, DeepSeek has proven that innovation knows no bounds.

At the same time, DeepSeek has security analysts, regulated companies, and those using AI models with sensitive data deeply concerned. As the AI race intensifies between the US and China, the implications of DeepSeek's rapid ascent extend beyond mere technological prowess. The platform's data collection practices have raised eyebrows, with reports of censorship and the potential for vast amounts of user data being sent back to China. This has led to concerns about privacy, security, and the broader geopolitical implications of the AI arms race. While the excitement surrounding DeepSeek is palpable, it is crucial for users to weigh the benefits against the potential risks, especially when it comes to sensitive information.

But… There is so much confusion around the security risks of a model like this. Is it actually possible to use DeepSeek privately? Can those operating in regulated or security/ privacy sensitive contexts leverage the power of these models in their own applications? Does anyone that uses the DeepSeek R1 model have to send their data to Chinese companies? We will answer those questions and help you think about similar issues for future-released models in this article.

How can you access the DeepSeek (and other AI) models?  

There are two primary ways that you can access and build on top of AI models like DeepSeek or many other models like Llama 3.3, OpenAI’s o1, etc.

1. Managed services or applications: Just like other AI models from providers like OpenAI or Meta, you can access the model through a managed platform or application owned and operated by the model provider (in this case DeepSeek). For DeepSeek this is a chat interface in the app store or at https://chat.deepseek.com. For OpenAI models this is ChatGPT or maybe Microsoft CoPilot. 

2. Self-hosted model endpoints: DeepSeek has also released their models openly on a model repository run by the company Hugging Face (which is the central repository for models and data, and is not associated in any way with DeepSeek). Models, including DeepSeek, can be downloaded from Hugging Face and run in your own infrastructure, including your cloud VPC, on-prem datacenter, or even your laptop (if you have enough power).

In the first scenario, DeepSeek (just like OpenAI with ChatGPT) is productizing the “model” into a service/application. This is not just the model, but it includes UI components, monitoring, data storage, and all the things that go along with operating a modern Software as a Service offering (SaaS). Any SaaS offering, from ChatGPT to AirBnB, has these components, and they don’t often share exactly what is happening behind the scenes of this product. It is their commercial offering, and they pre-process, store, cache, log, and post-process the data you provide in all sorts of ways that you can’t control.

The problem with DeepSeek’s managed service/app offering is that it is being operated by a Chinese entity, which brings us national security and privacy concerns. According to DeepSeek's own privacy policy for this service/app, the company (DeepSeek) collects large amounts of personal information from those using their managed product offering (which is stored "in secure servers" in China). We agree generally with those who have concerns related to the use of the managed services and applications, like this one, where the use and storage of your data is suspect.

However, in the second scenario (the self-hosted model endpoint scenario), you are not accessing the model via anything managed by DeepSeek. You are only running the “model” provided to them through a non-DeepSeek model repository (Hugging Face). So, that begs the question: can self-hosting DeepSeek be done securely? If so, what should you consider as related to this model compared with other, “domestic” models like Llama 3.1? To answer that, we first need to clarify what we mean when we refer to a “model”.

What constitutes an AI model like DeepSeek R1?

A modern AI model like DeepSeek R1 or Llama 3.1 has two major components:

1. Code (or software): This code loads the model parameters (see below) and accomplishes a data transformation of your inputs into the model outputs. The code takes your input “prompts” and uses software functions and loaded parameters to generate outputs (e.g., text generations).

2. Parameters: These parameters are needed to run the code (see above), and there are billions of them (in the case of DeepSeek around 700B). These are what is created in the “training” process of the model. They don’t include any raw information or data/ knowledge. They just serve to help the code generate probable text outputs (like autocomplete).

That’s it. That’s what is needed to run DeepSeek R1 in isolation. Code and Parameters. Of course, that doesn’t totally answer the question we posed above. Is it safe to utilize the DeepSeek code and parameters by downloading them to your infrastructure and running them? More generally, how would you know this is safe or risky for any AI model? 

How can you self-host a model like DeepSeek R1 securely?

Well, the code component of DeepSeek and other popular models isn’t actually code maintained by DeepSeek. It is part of an open source software project called “transformers”: https://github.com/huggingface/transformers (or at least that is the most popular way to run models like these). The code is available on GitHub and you can scan every line of it if you need to do something in your spare time. This means that the code can be analyzed with security tools and run in environments, such as air-gapped or “behind the firewall” environments, that are not even able to send data to China (or anywhere else for that matter). It is true that the transformers library allows third party code to be executed when support for a model is not yet merged into the upstream transformers project, but this is not the case with DeepSeek. No third party code is needed.

Regarding the parameters of the DeepSeek model, these are again provided on the Hugging Face platform as downloadable data files. Certain security risks have been identified related to the download and deserialization of similar data files (see, for example, this article from JFrog). There are certain secure and insecure formats of these data files that could open up security vulnerabilities. Thankfully, the “Distill” versions of DeepSeek R1 and other popular model providers are using the secure format of “safetensors” and no other third party code is needed to load the parameters of the model.

Note: As of the writing of this blog post the non-Distill “DeepSeek R1” model (the largest one of the variants released) does require third party code execution to load in the model using transformers. This will likely change very quickly (like in days or weeks) as the code is accepted into the upstream transformers project. Until then, it would be our recommendation to avoid using this model in self-hosted environments and leverage the “Distill” versions of the models, which don’t require this potential vulnerability.

To bring everything together and for clarity let’s answer of the questions we (or others) have raised for this model:

• If you are self-hosting an open access DeepSeek R1 model, will your data be sent to and stored in the infrastructure of Chinese companies? If you are using the “Distill” versions of the model (or when the full R1 code is merged into upstream transformers), No. And just to make sure, you can set up your firewalls rules and networking to prevent outbound traffic from model servers (like we do in the secure Prediction Guard platform).

• Is China or DeepSeek monitoring my use of DeepSeek R1? If you are using their managed services/application, then 100% yes. If you are self-hosting the “Distill” versions of the model (or when the full R1 code is merged into upstream transformers), then 100% no (unless your infrastructure team opens up other back doors in your infrastructure that have nothing to do with the model, and China happens to be interested in you).

• Should I use DeepSeek models in security/ privacy sensitive applications? You should not utilize a managed API or application from DeepSeek if you don’t trust the way they are running this or if you don’t agree with their privacy policy. However, you can self-host these models securely and privately and leverage their power. 

If I can run DeepSeek securely, is there any real concern?

We have mostly talked about the privacy concerns related to DeepSeek. These can be addressed by self-hosting the model or running the model in a self-hosted platform that you can control and configure. We’ve built the Prediction Guard platform, for example, to security scan and lock down models running in your infrastructure, such that supply chain vulnerabilities and insecure model hosting is addressed out-of-the-box.

However, leaks of information to China aren’t the only potential vulnerabilities or issues when trying to build secure, private AI applications. We detail some of the other things you might want to be thinking about in the article: System level security for open source AI models. Please reach out to us if you want this kind of system level security as you are trying to run models like DeepSeek. You can deploy Prediction Guard in your company infrastructure to enable magical AI functionality without sacrificing security/ privacy. Book a demo here.